search

p4lang / p4-dpdk-target

P4 driver SW for P4 DPDK target.
Apache License 2.0
57 stars 19 forks source link

Issue with running bfswitchd #89

Closed guy-davidi closed 1 year ago

guy-davidi commented 1 year ago

Hi, after changing the .json to:

Ports: `{ "ports": [ { "dev_port": 3, "port_name": "ens3", "mempool_name": "MEMPOOL0", "pipe_name": "pipe", "pipe_in":"pipe", "pipe_out":"pipe", "port_dir": "default", "port_in_id": 0, "port_out_id": 0, "port_type": "tap", "tap_port_attributes": { "mtu": 1500 } }, { "dev_port": 5, "port_name": "ens5", "mempool_name": "MEMPOOL0", "pipe_name": "pipe", "port_dir": "default", "port_in_id": 1, "pipe_in":"pipe", "pipe_out":"pipe", "port_out_id": 1, "port_type": "tap", "tap_port_attributes": { "mtu": 1500 } }

]

}`

switch config:

{ "chip_list": [ { "id": "asic-0", "chip_family": "dpdk", "instance": 0 } ], "instance": 0, "p4_devices": [ { "device-id": 0, "eal-args": "-n 1 -c 0x1", "mempools": [ { "name": "MEMPOOL0", "buffer_size": 2304, "pool_size": 1024, "cache_size": 256, "numa_node": 0 } ], "p4_programs": [ { "program-name": "l1switch", "bfrt-config": "/home/p4project/l1switch/tdi.json", "port-config": "/home/p4project/l1switch/ports.json", "p4_pipelines": [ { "p4_pipeline_name": "pipe", "core_id": 1, "numa_node": 0, "context": "/home/p4project/l1switch/context.json", "config": "/home/p4project/l1switch/main.spec", "pipe_scope": [ 0, 1, 2, 3 ], "path": "/home/p4project/l1switch/" } ] } ] } ] }

build.sh :

`#!/bin/bash p4c-dpdk main.p4 \ -o main.spec \ --arch psa \ --bf-rt-schema main.bfrt.json \ --context context.json \ --p4runtime-files main.pb.txt \ --p4runtime-format text \ --tdi tdi.json

Basic environment variables we need

export SDE=$HOME/sde export SDE_INSTALL=$SDE/install export LD_LIBRARY_PATH=$SDE_INSTALL/lib:$SDE_INSTALL/lib64:$SDE_INSTALL/lib/x86_64-linux-gnu/:/usr/local/lib64:/usr/local/lib

We need to put SDE executable path before the system one since the bfrt_python

needs to use Python libraries from it and use the Python executable from SDE

export PATH=$SDE_INSTALL/bin:$PATH hash -r

For security reason, the PATH and LD_LIBRARY_PATH won't pass to root user even if we use "sudo -E"

We must pass them in sudo to make sure it is correct.

sudo -E PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH $SDE_INSTALL/bin/bf_switchd --install-dir $SDE_INSTALL --conf switch_config.json `

but sill see issues:

`p4project@p4project-S5520HC:~/l1switch$ ./build.sh Install dir: /home/p4project/sde/install (0x564c18724c20) bf_switchd: system services initialized bf_switchd: loading conf_file switch_config.json... bf_switchd: processing device configuration... Configuration for dev_id 0 Family : dpdk bf_switchd: processing P4 configuration... P4 profile for dev_id 0 P4 EAL args -n 1 -c 0x1 Debug CLI disable num mempool objs 1 mempool_name: MEMPOOL0 buffer_size: 2304 pool_size: 1024 cache_size: 256 numa_node: 0 num fixed functions 0 num P4 programs 1 p4_name: l1switch bfrt_config: /home/p4project/l1switch/tdi.json port_config: /home/p4project/l1switch/ports.json p4_pipeline_name: pipe core_id: 1 numa_node: 0 context: /home/p4project/l1switch/context.json config: /home/p4project/l1switch/main.spec Pipes in scope [0 1 2 3 ] Timer values [] Mirror Config n_slots: 4 n_sessions: 64 fast_clone: 0 diag: accton diag: non_default_port_ppgs: 0 SAI default initialize: 1 Operational mode set to DPDK MODEL Initialized the device types using PAL handler registration STUB:port_mgr_platform_init DPDK bf_switchd: drivers initialized EAL: Detected CPU lcores: 16 EAL: Detected NUMA nodes: 2 EAL: Detected shared linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: VFIO support initialized EAL: DPDK is running on a NUMA system, but is compiled without NUMA support. EAL: This will have adverse consequences for performance and usability. EAL: Please use --legacy-mem option, or recompile with NUMA support. EAL: Device 0000:01:00.0 is not NUMA-aware EAL: Device 0000:01:00.1 is not NUMA-aware EAL: Device 0000:03:00.0 is not NUMA-aware EAL: Using IOMMU type 8 (No-IOMMU) EAL: Ignore mapping IO port bar(2) EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:03:00.0 (socket -1) EAL: Device 0000:07:00.0 is not NUMA-aware EAL: Ignore mapping IO port bar(2) EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:07:00.0 (socket -1) TELEMETRY: No legacy callbacks, legacy socket not created

bf_switchd: dev_id 0 initialized

bf_switchd: initialized 1 devices bf_switchd_switchsai_lib_init starting SAI bf_switchd: spawning cli server thread bf_switchd: spawning driver shell bf_switchd: server started - listening on port 9999

    ********************************************
    *      WARNING: Authorised Access Only     *
    ********************************************

bfshell> bfrt_python cwd : /home/p4project/l1switch

We've found 1 p4 programs for device 0: l1switch Creating tree for dev 0 and program l1switch

Devices found : [0] /home/p4project/sde/install/lib/python3.10/site-packages/IPython/core/history.py:226: UserWarning: IPython History requires SQLite, your history will not be saved warn("IPython History requires SQLite, your history will not be saved") Python 3.10.8 (aaaf517424:aaaf517424, Dec 27 2022, 12:59:27) [GCC 11.3.0] Type 'copyright', 'credits' or 'license' for more information IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: bfrt ------> bfrt() Available symbols: dump - Command enable - Command info - Command l1switch - Node port - Node

bfrt> l1switch.enable ----> l1switch.enable() 2022-12-28 18:27:04.276807 BF_PIPE ERROR - error :thread pipeline enable 2022-12-28 18:27:04.276956 BF_PIPE ERROR - Failed to Build Pipeline pipe 2022-12-28 18:27:04.277010 BF_BFRT ERROR - bf_rt_enable_pipeline:120 failed `

What should we write in switch_config.json and ports.json? We would appreciate a link to a full tutorial \ guide on how to write these .json files.

Thank you

qobilidop commented 1 year ago

Hi @guy-davidi, how did you solve this? I'm seeing similar issues.

guy-davidi commented 1 year ago

Hi unfortunately no

qobilidop commented 1 year ago

My temporary workaround is this.

ShatakshiMishra commented 10 months ago

Hi, I was was trying to run the P4-DPDK-Target example, and encounter some similar kind of error. Please suggest what could be the possible reason behind this error.

switch_conf.json

{ "chip_list": [ { "id": "asic-0", "chip_family": "dpdk", "instance": 0 } ], "instance": 0, "p4_devices": [ { "device-id": 0, "eal-args": "dummy -n 4 -c 3", "mempools": [ { "name": "MEMPOOL0", "buffer_size": 2304, "pool_size": 1024, "cache_size": 256, "numa_node": 0 } ], "p4_programs": [ { "program-name": "pna", "bfrt-config": "/home/p4dpdk/install/share/dpdk/pna/bf-rt.json", "p4_pipelines": [ { "p4_pipeline_name": "pipe", "core_id": 1, "numa_node": 0, "context": "/home/p4dpdk/install/share/dpdk/pna/pipe/context.json", "config": "/home/p4dpdk/install/share/dpdk/pna/pipe/simple_lpm.spec", "pipe_scope": [ 0, 1, 2, 3 ], "path": "/home/p4dpdk/install/share/dpdk/pna" } ] } ] } ] }

:~/p4dpdk/install/bin$ _sudo -E PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH $SDE_INSTALL/bin/bf_switchd --install-dir $SDE_INSTALL --conf-file $SDE_INSTALL/share/dpdk/pna/switchconf.json --init-mode=cold --status-port 7777

Install dir: /home/p4dpdk/install (0x562661a53c80) bf_switchd: system services initialized bf_switchd: loading conf_file /home/p4dpdk/install/share/dpdk/pna/switch_conf.json... bf_switchd: processing device configuration... Configuration for dev_id 0 Family : dpdk bf_switchd: processing P4 configuration... P4 profile for dev_id 0 P4 EAL args dummy -n 4 -c 3 Debug CLI disable num mempool objs 1 mempool_name: MEMPOOL0 buffer_size: 2304 pool_size: 1024 cache_size: 256 numa_node: 0 num fixed functions 0 num P4 programs 1 p4_name: pna bfrt_config: /home/p4dpdk/install/share/dpdk/pna/bf-rt.json port_config: p4_pipeline_name: pipe core_id: 1 numa_node: 0 context: /home/p4dpdk/install/share/dpdk/pna/pipe/context.json config: /home/p4dpdk/install/share/dpdk/pna/pipe/simple_lpm.spec Pipes in scope [0 1 2 3 ] Timer values [] Mirror Config n_slots: 4 n_sessions: 64 fast_clone: 0 diag: accton diag: non_default_port_ppgs: 0 SAI default initialize: 0 Operational mode set to DPDK MODEL Initialized the device types using PAL handler registration STUB:port_mgr_platform_init DPDK bf_switchd: drivers initialized EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Detected shared linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' TELEMETRY: No legacy callbacks, legacy socket not created

bf_switchd: dev_id 0 initialized

bf_switchd: initialized 1 devices bf_switchd_switchsai_lib_init starting SAI bf_switchd: spawning cli server thread bf_switchd: spawning driver shell bf_switchd: server started - listening on port 9999

    ********************************************
    *      WARNING: Authorised Access Only     *
    ********************************************

bfshell> bfrt_python cwd : /home/p4dpdk/install/bin

We've found 1 p4 programs for device 0: pna Creating tree for dev 0 and program pna

Devices found : [0] /home/p4dpdk/install/lib/python3.10/site-packages/IPython/core/history.py:226: UserWarning: IPython History requires SQLite, your history will not be saved warn("IPython History requires SQLite, your history will not be saved") Python 3.10.9 (1dd9be6584:1dd9be6584, Dec 13 2023, 10:31:12) [GCC 10.5.0] Type 'copyright', 'credits' or 'license' for more information IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: bfrt ------> bfrt() Available symbols: dump - Command enable - Command info - Command pna - Node port - Node

bfrt> pna.enable ----> pna.enable() _2023-12-13 15:22:32.227229 BF_PIPE ERROR - dal_enable_pipeline line:165 Pipeline build failed 2023-12-13 15:22:32.227266 BF_PIPE ERROR - Failed to Build Pipeline pipe 2023-12-13 15:22:32.227278 BF_BFRT ERROR - bf_rt_enable_pipeline:120 failed_

bfrt>