ES2K Setup Guide

This document explains how to build, install, and run P4 Control Plane for the ES2K target.

P4 Control Plane can be built to run on an x86 host processor or the ARM Compute Complex (ACC). These instructions are for the host processor. For the ACC, see Building for the ES2K ACC.

Prerequisites

It is assumed that the ES2K IPU SDE dependencies have been installed on the system. The list of dependencies can be found in sde/tools/setup/install_dep.py file.

Prerequisites for stratum-deps are assumed to be installed. Some of them also apply to networking-recipe.

Install the following packages for building networking-recipe.

For Fedora:

yum install libatomic libnl3-devel
pip3 install -r requirements.txt

For Ubuntu:

apt install libatomic1 libnl-route-3-dev
pip3 install -r requirements.txt

See the OpenSSL security guide for information on installing OpenSSL.

Build and install ES2K SDE

Obtain a copy of the IPU SDK (SDE) for the Intel® IPU E2100 from the manufacturer, together with the instructions for building or installing it.

The above description is intentionally vague. The ES2K SDE (now called the IPU SDK) is under active development. This document cannot keep pace with the changes.

Once installed, you will need to locate the root directory of the host SDE. A typical location is /opt/p4/p4sde.

You can use the Linux file command to verify that the file structure is for the x86-64 architecture.

ahab@pequod:~/latest$ file -L /opt/deps/bin/protoc
/opt/deps/bin/protoc: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
BuildID[sha1]=93b21eff920e615fee679b0bd0a3ecc0c1139575, for GNU/Linux 3.2.0,
not stripped

Remember the directory in which you install the ES2K SDE. You will need it to define the SDE_INSTALL environment variable.

Build and install stratum dependencies

Clone the repository used to build the Stratum dependencies:

git clone --recursive https://github.com/ipdk-io/stratum-deps.git

Now follow the instructions in the Building Host Dependencies document.

Remember the directory in which you install the Stratum dependencies. You will need it to define the DEPEND_INSTALL environment variable.

Define environment variables

Define the following environment variables. They supply default values to the build system and helper scripts.

Variable

Definition

DEPEND_INSTALL

Path to the directory in which the Stratum dependencies are installed.

SDE_INSTALL

Path to the directory in which the ES2K SDE is installed.

Build P4 Control Plane

Clone repository

Clone the repository used to build P4 Control Plane:

git clone --recursive https://github.com/ipdk-io/networking-recipe.git p4cp.recipe
cd p4cp.recipe
export P4CP_RECIPE=`pwd`

Compile the recipe

cd $P4CP_RECIPE
./make-all.sh --target=es2k --rpath

By default, make-all.sh will create an install folder in the networking-recipe directory in which to install the build artifacts. You can specify a different directory by means of the --prefix parameter.

See the make-all.sh user guide for information about the options of the make-all.sh helper script.

Define P4CP_INSTALL

We recommend that you define the following environment variable:

Variable

Definition

P4CP_INSTALL

Path to the directory in which the P4 Control Plane build artifacts are installed.

It is used throughout the remainder of this document.

Run Infrap4d

Set up the environment required by infrap4d

Note: sudo is required when running copy_config_files.sh since you are copying files to system directories.

source $P4CP_INSTALL/sbin/setup_env.sh $P4CP_INSTALL $SDE_INSTALL $DEPEND_INSTALL
sudo $P4CP_INSTALL/sbin/copy_config_files.sh $P4CP_INSTALL $SDE_INSTALL

Set hugepages required for ES2K

Run the hugepages script.

sudo $P4CP_INSTALL/sbin/set_hugepages.sh

Export all environment variables to sudo user

alias sudo='sudo PATH="$PATH" HOME="$HOME" LD_LIBRARY_PATH="$LD_LIBRARY_PATH" SDE_INSTALL="$SDE_INSTALL"'

Modify the conf file

Update /usr/share/stratum/es2k/es2k_skip_p4.conf with CPF BDF as per your setup. User can get CPF BDF by running lspci | grep 1453 Note: max vport you can pass here from [0-6] “eal-args”: “–lcores=1-2 -a <cpf_bdf>,vport=[0-1] – -i –rxq=1 –txq=1 –hairpinq=1 –hairpin-mode=0x0”

Eg. “eal-args”: “–lcores=1-2 -a 00:01.6,vport=[0-1] – -i –rxq=1 –txq=1 –hairpinq=1 –hairpin-mode=0x0”

We give options to each process to request numbers of configure queues before starting the process

Admin must set cfgqs-idx between 0-15. See the latest es2k_skip_p4.conf for the exact syntax. If you want 16 cfgqs, use “cfgqs-idx”: “0-15”. Specify the range as a string. Index numbers from 0 to 15 are supported. In multi process environment, user should plan and split the queues between primary and secondary processes and specify the range with cfgqs-idx parameter. Total number of queues split between processes should not exceed 16.

Run the infrap4d daemon

sudo $P4CP_INSTALL/sbin/infrap4d

Note: By default, infrap4d runs in detached mode. If you want to run in attached mode, specify the --nodetach command-line option.

  • All infrap4d logs are by default logged under /var/log/stratum.

  • All P4SDE logs are logged in p4_driver.log under $P4CP_RECIPE.

  • All OVS logs are logged under /tmp/ovs-vswitchd.log.

Run a sample program

Open a new terminal to set the pipeline and try the sample P4 program. Set up the environment and export all environment variables to sudo user.

source $P4CP_INSTALL/sbin/setup_env.sh $P4CP_INSTALL $SDE_INSTALL $DEPEND_INSTALL
$P4CP_INSTALL/sbin/copy_config_files.sh $P4CP_INSTALL $SDE_INSTALL
alias sudo='sudo PATH="$PATH" HOME="$HOME" LD_LIBRARY_PATH="$LD_LIBRARY_PATH" SDE_INSTALL="$SDE_INSTALL"'

Create IPDF netdevs

After installing ATE Kernel on HOST machine, install the following drivers to bind the network devices (netdevs) to the E2100 target.

modprobe auxiliary 
modprobe idpf

Create P4 artifacts

  • P4 programs are available in the P4 SDE repository.

  • Obtain a copy of the P4 Compiler for ES2K and install it on your development system.

  • Set the environment variable OUTPUT_DIR to the location where artifacts should be generated and where p4 files are available

export OUTPUT_DIR=/root/p4-driver/mev_reference_p4_files/simple_l3_l4_pna

  • Generate the artifacts using the p4c compiler installed in the previous step:

p4c-pna-xxp -I/usr/lib -I/usr/share/p4c/p4include -I/usr/share/p4c/idpf-lib \
            $OUTPUT_DIR/simple_l3_l4_pna.p4 -o $OUTPUT_DIR/simple_l3_l4_pna.s \
            --p4runtime-files $OUTPUT_DIR/simple_l3_l4_pna.p4info.txt \
            --context $OUTPUT_DIR/simple_l3_l4_pna.context.json \
            --bfrt $OUTPUT_DIR/simple_l3_l4_pna.tdi.json

Note: The above commands will generate three files (simple_l3_l4_pna.p4info.txt, simple_l3_l4_pna.tdi.json, and simple_l3_l4_pna.context.json).

  • Modify simple_l3_l4_pna.conf to specify the correct paths for tdi-config, context, and config.

  • Create a dummy tofino.bin file, which is needed by tdi_pipeline_builder.

touch tofino.bin

  • TDI pipeline builder combines the artifacts generated by the p4c compiler to generate a single bin file to be pushed from the controller. Generate binary executable using the TDI pipeline builder command below:

$P4CP_INSTALL/bin/tdi_pipeline_builder \
    --p4c_conf_file=$OUTPUT_DIR/simple_l3_l4_pna.conf \
    --tdi_pipeline_config_binary_file=$OUTPUT_DIR/simple_l3_l4_pna.pb.bin

Set forwarding pipeline

sudo $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/simple_l3_l4_pna.pb.bin $OUTPUT_DIR/simple_l3_l4_pna.p4info.txt

Configure forwarding rule to receive traffic

Add a forwarding rule to receive traffic on VSI-1 (base offset 16 + VSI ID 1)
when the key matches.

sudo  $P4CP_INSTALL/bin/p4rt-ctl add-entry br0 MainControlImpl.l3_l4_match_rx \
    "hdrs.ipv4[vmeta.common.depth].protocol=0x11,vmeta.common.port_id=0,istd.direction=0,
    hdrs.ipv4[vmeta.common.depth].src_ip="192.168.1.10",hdrs.ipv4[vmeta.common.depth].dst_ip="192.168.1.20",
    hdrs.udp[vmeta.common.depth].sport=1000,hdrs.udp[vmeta.common.depth].dport=2000,action=MainControlImpl.send(17)"

See the p4rt-ctl client guide for more information on the p4rt-ctl utility.