search

tbarbette / fastclick

FastClick - A faster version of the Click Modular Router featuring batching, advanced multi-processing and improved Netmap and DPDK support (ANCS'15). Check the metron branch for Metron specificities (NSDI'18). PacketMill modifications (ASPLOS'21) as well as MiddleClick(ToN, 2021) are merged in main.
Other
274 stars 82 forks source link

Unknown or invalid PORT #189

Closed ligang626 closed 5 years ago

ligang626 commented 5 years ago

The current version of dpdk18.05 Fastclick using metron branch

execute command:bin/click --dpdk -c 0x03 -v -- conf/metron/metron-flow-director.click error: conf/metron/metron-flow-director.click:56: While configuring ‘fd1 :: FromDPDKDevice’: 1: Unknown or invalid PORT conf/metron/metron-flow-director.click:52: While configuring ‘fd0 :: FromDPDKDevice’: 0: Unknown or invalid PORT conf/metron/metron-flow-director.click:54: While configuring ‘ToDPDKDevice@5 :: ToDPDKDevice’: 0 : Unknown or invalid PORT conf/metron/metron-flow-director.click:58: While configuring ‘ToDPDKDevice@8 :: ToDPDKDevice’: 1 : Unknown or invalid PORT conf/metron/metron-flow-director.click:39: While configuring ‘metron :: Metron’: Rx filter mode: flow

tbarbette commented 5 years ago

Hi,

Did you set up some ports to work with DPDK?

You should execute that command as root, also. What NIC do you have?

Tom

ligang626 commented 5 years ago

How to set up DPDK port? I am using the root user

gang.li@raymax.net

From: Tom Barbette Date: 2019-08-02 16:14 To: tbarbette/fastclick CC: ligang626; Author Subject: Re: [tbarbette/fastclick] Unknown or invalid PORT (#189) Hi, Did you set up some ports to work with DPDK? You should execute that command as root, also. What NIC do you have? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

tbarbette commented 5 years ago

DPDK uses user-level drivers. Depending on your NIC you may need to load the igb_uio module and re-bind the NIC to the UIO driver. More info there: https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html

gkatsikas commented 5 years ago

Any progress on this?

In your DPDK tree, try 'python usertools/dpdk-devbind.py' to see which NICs are available in your system. You could post the output here to get some help.

ligang626 commented 5 years ago

The rest device and the ports in the device are already visible in onos. I am researching how to bind the rest device to p4 (onos-apps-p4-turorial-pipeconf) in the pipeconf.

gkatsikas commented 5 years ago

This is not possible as far as I know. P4 pipeconf is a different driver for P4 capable devices; servers with normal NICs are not such devices. If you see the server's NIC in ONOS, this is because of the server device driver.

ligang626 commented 5 years ago

What our company is doing now is to support only the network card of p4. Dpdk is also adding our NIC driver

gkatsikas commented 5 years ago

Nice.. Which driver are you extending on the ONOS side?

ligang626 commented 5 years ago

Currently working on the onos-apps-p4tutorial project and the code related to pipeconf Now only onos devices can see Intel's network card, our company's network card, I am still debugging If you have better suggestions, please let me know, thank you.

ligang626 commented 5 years ago

Network devices using DPDK-compatible driver

0000:01:00.1 'I350 Gigabit Network Connection 1521' drv=igb_uio unused=igb,uio_pci_generic 0000:84:00.0 'Device 903f' drv=igb_uio unused=uio_pci_generic

You can see two network cards in dpdk The first one is Intel and the second one is our company. Use the following command to start click Bin/click --dpdk -c 0x03 -v -- conf/metron/metron-flow-director.click Will report an error:    1: Unknown or invalid PORT Initializing flow parser... Conf/metron/metron-flow-director.click:52: While configuring ‘fd0 :: FromDPDKDevice’:    Warning: Flow Director (port 0): FLOW_DIR_RULES_FILE is not set, hence this NIC can only be configured by the handlers    Flow Director (port 0): State active - Source file 'None' Conf/metron/metron-flow-director.click:58: While configuring ‘ToDPDKDevice@8 :: ToDPDKDevice’:    1 : Unknown or invalid PORT Conf/metron/metron-flow-director.click:39: While configuring ‘metron :: Metron’:    Rx filter mode: flow

But if you comment out the two lines of "fd1" in conf/metron/metron-flow-director.click, you can start it. But only see one network card

Does not rule out the problem of the driver in our dpdk

gkatsikas commented 5 years ago

From your logs above, I guess that the second NIC is bound to a generic DPDK driver because no other driver is suitable. Are you sure that the vendor of this second NIC is DPDK-supported?

This is why commenting fd1 and td1 works for you, but only with one NIC (fd0 and td0)

tbarbette commented 5 years ago

It seems both driver are not bound, can you run ./usertools/dpdk-devbind.py --bind=uio_pci_generic 01:00.1 and ./usertools/dpdk-devbind.py --bind=uio_pci_generic 84:00.0 However, one may be your management interface... If the machine has one dataplane interface, only bind that one. And comment "fd1" and "td1" references.

I'm not sure to get the whole driver story...? If the driver is not supported by DPDK you can use an "emulated" one with AF_PACKET, using good old sockets. However Metron needs at least the ability to classify on ethernet fields, and AF_PACKET does not provide that. Depends on what is your end goal... The SoftNIC DPDK driver may help you implement missing features in software, but you loose the no-CPU dispatching advantage of Metron. If you're just trying to "link" the dataplane with ONOS, this is fine.

ligang626 commented 5 years ago

./usertools/dpdk-devbind.py --bind=uio_pci_generic 01:00.1 I tried to run this. Still prompted that the port does not exist. Our R&D staff said that he will debug later.

Now my question is: Is there a way to bind this rest device to a pipeconf? "rest:192.168.7.40:80":          {              "rest":              {                  "username": "server",                  "password": "",                  "ip": "192.168.7.40",                  "port": 80,                  "protocol": "http",                  "url": "",                  "testUrl": "",                  "manufacturer": "GenuineIntel",                  "hwVersion": "Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz",                  "swVersion": "Click 2.1"              },              "basic":              {                  "driver": "restServer"              },              "piPipeconf": {                   "piPipeconfId": "p4-tutorial-pipeconf"              }          }

I wrote it like this. But it doesn't seem to work. Currently studying the source code of onos

gkatsikas commented 5 years ago

This won't work due to several reasons:

1) What is your server's HW architecture? Do an: 'lscpu' to see the details and set the 'manufacturer' and 'hwVersion' fields respectively. The 'manufacturer' field can be 'GenuineIntel' or 'AMD', while the 'hwVersion' field should be filled according to the outcome of the 'lscpu' command. If your CPU is 'Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz' then leave the above configuration as it is, but a Xeon Gold could have a different configuration there. Once you fill the above fields correctly, modify the same fields (manufacturer and hwVersion) in the onos/drivers/server/src/main/resources/server-drivers.xml file and then start ONOS with this driver enabled.

2) Remove the whole "pipeconf" : {} json configuration as the server device driver does not recognize this configuration.

ligang626 commented 5 years ago

Feel that using the rest device and pipeconf binding is not feasible It seems to be using the general device, but I don't know the port of p4runtime.

"device:identifier": {        "generalprovider": {          "p4runtime": {            "ip": "192.168.56.104",            "port": 830,            "deviceKeyId": "p4runtime:device:identifier"          },          "gnmi": {            "ip": "192.168.56.105",            "port": 830,            "deviceKeyId": "p4runtime:device:identifier"          }        },        "piPipeconf":{          "piPipeconfId": "pipipeconfTest"        },        "basic": {          "driver": "bmv2"        }      },

gkatsikas commented 5 years ago

For the p4runtime you need to ask Carmelo Cascone at the onos Dev list. He is the expert in P4. I recall that p4runtime uses another southbound protocol to talk to p4 capable devices

cxxuser commented 2 years ago

I have met the same problem: EAL: Detected 32 lcore(s) EAL: Detected 2 NUMA nodes EAL: RTE Version: 'DPDK 20.05.0-rc4' EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: No free hugepages reported in hugepages-2048kB EAL: Probing VFIO support... EAL: VFIO support initialized EAL: No legacy callbacks, legacy socket not created CONFIG-1:2: While configuring ‘FromDPDKDevice@1 :: FromDPDKDevice’: 0000:04:00.1: Unknown or invalid PORT CONFIG-1:3: While configuring ‘ToDPDKDevice@2 :: ToDPDKDevice’: BURST: unknown argument Router could not be initialized!

My CONFIG-1 is very simple,only one line codes: FromDPDKDevice(0000:04:00.1, N_QUEUES 1, MAXTHREADS 1, PAUSE none, BURST 32, NDESC 128, THREADOFFSET 0, NUMA false)-> ToDPDKDevice(0000:04:00.1, IQUEUE 1, BLOCKING true, BURST 32);

AND this is what my dpdk-devbind.py --status shows: Network devices using DPDK-compatible driver

0000:04:00.1 'MT416842 BlueField integrated ConnectX-5 network controller a2d2' drv=uio_pci_generic unused=mlx5_core

Network devices using kernel driver

0000:01:00.0 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe 165f' if=em1 drv=tg3 unused=uio_pci_generic Active 0000:01:00.1 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe 165f' if=em2 drv=tg3 unused=uio_pci_generic 0000:02:00.0 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe 165f' if=em3 drv=tg3 unused=uio_pci_generic 0000:02:00.1 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe 165f' if=em4 drv=tg3 unused=uio_pci_generic 0000:04:00.0 'MT416842 BlueField integrated ConnectX-5 network controller a2d2' if=p6p1 drv=mlx5_core unused=uio_pci_generic

I have binded my mlx5 to the uio_pci_generic, and as far as I know,the mlx can also work on kernel driver. However ,whatever I bind the mlx5, I can only get the terrible results：“While configuring ‘FromDPDKDevice@1 :: FromDPDKDevice’: 0000:04:00.1: Unknown or invalid PORT”