Vhost config for netronome-packetgen

[sarsanaee]

Hello

I want to config the netronome-packetgen to work with a vhost PMD port(using a vdev parameter), however, I have some issues in configuring it.

Here is the command I run from the MOONGEN_ROOT: sudo ./build/MoonGen examples/netronome-packetgen/packetgen.lua --dpdk-config=libmoon/dpdk-conf.lua --tx-slave 0

Here is my dpdk-config:

DPDKConfig {
    -- configure the CPU cores to use, default: all cores
    --cores = {0, 10, 11, 12, 13, 14, 15},
    cores = {0, 1},

    -- max number of shared tasks running on core 0
    sharedCores = 2,

    -- black or whitelist devices to limit which PCI devs are used by DPDK
    -- only one of the following examples can be used
    pciBlacklist = {"0000:81:00.0","0000:81:00.1"},
    --pciWhitelist = {"0000:81:00.3","0000:81:00.1"},
    --
    -- arbitrary DPDK command line options
    -- the following configuration allows multiple DPDK instances (use together with pciWhitelist)
    -- cf. http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html#running-multiple-independent-dpdk-applications
    cli = {
        --"--file-prefix", "m1",
        "--socket-mem", "1024,1024",
        "--log-level", "8",
        --"--proc-type", "auto",
        "--vdev", "virtio_user0,path=/users/alireza/my_vhost1.sock,queues=1"
    }

}

It actually recognizes my vhost port, here is the dpdk log:

EAL: TSC frequency is ~1995388 KHz
EAL: Master lcore 0 is ready (tid=b1181940;cpuset=[0])
EAL: lcore 3 is ready (tid=ad193700;cpuset=[0])
EAL: lcore 2 is ready (tid=ad994700;cpuset=[0])
EAL: lcore 1 is ready (tid=ae195700;cpuset=[1])
EAL: PCI device 0000:81:00.0 on NUMA socket 1
EAL:   Device is blacklisted, not initializing
EAL: PCI device 0000:81:00.1 on NUMA socket 1
EAL:   Device is blacklisted, not initializing
EAL: Search driver virtio_user0 to probe device virtio_user0
[INFO]  Found 1 usable devices:
   Device 0: CA:53:6D:1B:E3:34 (unknown NIC (PCI ID 7fcf:0))

However, I still get the following error:

[FATAL] Lua error in task master
/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/device.lua:196: Could not configure device 0: error Operation not supported
Stack Traceback
===============
(2) Lua method 'fatal' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/log.lua:129'
    Local variables:
     self = table: 0x400bff98  {DEBUG:0, fatal:function: 0x40a98848, writeToLog:function: 0x40a98828, INFO:1 (more...)}
     str = string: "Could not configure device 0: error Operation not supported"
(3) Lua field 'config' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/device.lua:196'
    Local variables:
     args = table: 0x41704020  {rxQueues:1, stripVlan:true, dropEnable:true, mempools:table: 0x40837a58 (more...)}
     driverInfo = table: 0x40a573d8  {}
     rc = number: -95
(4) Lua function 'master' at file 'examples/netronome-packetgen/packetgen.lua:250' (best guess)
    Local variables:
     continue = boolean: true
     totalSlaves = number: 1
     portList = table: 0x41703fb0  {1:0}
     devices = table: 0x41703ff8  {}
     (for generator) = C function: builtin#6
     (for state) = table: 0x41703fb0  {1:0}
     (for control) = number: 1
     _ = number: 1
     port = number: 0
     deviceIdx = number: 1
(5) global C function 'xpcall'
(6) Lua upvalue 'master' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/main.lua:96'
    Local variables:
     _ = string: "./build/MoonGen"
     file = string: "examples/netronome-packetgen/packetgen.lua"
     args = table: 0x419b85d0  {1:--tx-slave, 2:0}
     cfgFile = nil
     ok = boolean: true
     parsedArgs = table: 0x40e6a930  {}
(7) Lua function 'main' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/main.lua:146' (best guess)
    Local variables:
     task = string: "master"
    /proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/device.lua:196: Could not configure device 0: error Operation not supported

I couldn't figure out where the issue is in this case!

Thanks, Alireza

[emmericp]

That error means its trying to configure some feature not supported by the (virtual) device, I unfortunately don't know much about the netronome packetgen and vhost devices, so I'm afraid it could be literally anything.

Two ways to debug this:

1) start with a simpler script, try the disableOffloads option

2) figure out where exactly in device.c it is failing

[NetronomeMoongen]

Hi,

The Netronome packetgen script (by default), uses 1 TX queue, 1 RX queue, 1023 TX descriptors and 1023 RX descriptors when configuring a device.

Since in your case you did not specify any additional arguments, tose are the values provided to device.lua for the device configuration (all other values are left unspecified / nil)

I agree with Paul that the issue occurs when MoonGen / device configuration is taking place, and one or more of the settings are not supported by your device.

.

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, Aug 13, 2019 at 1:41 PM Paul Emmerich notifications@github.com wrote:

That error means its trying to configure some feature not supported by the (virtual) device, I unfortunately don't know much about the netronome packetgen and vhost devices, so I'm afraid it could be literally anything.

Two ways to debug this:

1.

start with a simpler script, try the disableOffloads option 2.

figure out where exactly in device.c it is failing

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/253?email_source=notifications&email_token=AEX76NMAFXN4QZ7LH3TDKS3QEKMW3A5CNFSM4IKCLIMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FMR6Y#issuecomment-520800507, or mute the thread https://github.com/notifications/unsubscribe-auth/AEX76NOF2663DDBE6RLGJQLQEKMW3ANCNFSM4IKCLIMA .

[sarsanaee]

Hello,

Thank you for the response, the problem was the disableOffload, which is false by default. For vdev devices, I think we should set it true somewhere.

[immadhavv]

@sarsanaee @emmericp Hi, I'm trying to set up a system where MoonGen(in a container) send line rate traffic to testpmd(in a container) through virtual ports (dpdkvhostuser type) connected to an ovs-dpdk bridge. I have been able to initialize one vdev parameter like you've mentioned here, and its getting recognized, but I'm not sure how to add more than one vdev parameter.

Can someone help me understand how this dpdk-config.lua file works and how to manipulate it and make use of it for my purpose?

[NetronomeMoongen]

Hi,

I'm not sure what exactly you are having trouble with, but from the packetgen script point of view, here are some pointers:

You can specify multiple tx and / or rx ports / slaves on the command line using the -tx and -rx command line parameters. For example, to specify using ports 0 and 1 for tx, and ports 2 and 3 for rx, use these parameters:

-tx 0 -tx 1 -rx 2 -rx 3

The port numbers refer to the ports / interface / devices as they are defined in the dpdk conf .lua file.

As for configuring multiple VHost devices in a dpdk conf file, I'm afraid I am not too familiar with how to do that.

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Apr 15, 2020 at 9:28 AM Niranjan Madhavan notifications@github.com wrote:

@sarsanaee https://github.com/sarsanaee @emmericp https://github.com/emmericp Hi, I'm trying to set up a system where MoonGen(in a container) send line rate traffic to testpmd(in a container) through virtual ports (dpdkvhostuser type) connected to an ovs-dpdk bridge. I have been able to initialize one vdev parameter like you've mentioned here, and its getting recognized, but I'm not sure how to add more than one vdev parameter.

Can someone help me understand how this dpdk-config.lua file works and how to manipulate it and make use of it for my purpose?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/253#issuecomment-613866644, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEX76NO6K572YHZ4LRZP23DRMVOYBANCNFSM4IKCLIMA .

[immadhavv]

Hi @NetronomeMoongen Thanks for the pointers. Actually, I've already been able to send packets from pktgen using the similar setup, I've mentioned before.

But now I want to send using MoonGen , as I need it to resemble a DDOS attack. I've been able to configure one Vhost device in the dpdk-config file like @sarsanaee mentioned. It's getting recognized as Device 0. But I need to configure another vhost device too as Device 1 so that I can send packets.

[NetronomeMoongen]

Hi,

Understood. I'm afraid I can't help with the VHost config stuff.

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Apr 15, 2020 at 10:22 AM Niranjan Madhavan notifications@github.com wrote:

Hi @NetronomeMoongen https://github.com/NetronomeMoongen Thanks for the pointers. Actually, I've already been able to send packets from pktgen using the similar setup, I've mentioned before.

But now I want to send using MoonGen , as I need it to resemble a DDOS attack. I've been able to configure one Vhost device in the dpdk-config file like @sarsanaee https://github.com/sarsanaee mentioned. It's getting recognized as Device 0. But I need to configure another vhost device too as Device 1 so that I can send packets.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/253#issuecomment-613893659, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEX76NI7B7UE7W7Q7BI5SYTRMVVEPANCNFSM4IKCLIMA .

[sarsanaee]

Adding multiple --vdev ports means adding multiple --vdev config to your DPDK configs I suppose. I still don't quite well understand what the problem is. Is this all on one node?

You can just pass dpdk-config file to moongen like --dpdk-config="lalal"

Hope that helps, Alireza

On Wed, Apr 15, 2020 at 4:36 PM Netronome-moongen notifications@github.com wrote:

Hi,

Understood. I'm afraid I can't help with the VHost config stuff.

< https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail

Virus-free. www.avast.com < https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Apr 15, 2020 at 10:22 AM Niranjan Madhavan < notifications@github.com> wrote:

Hi @NetronomeMoongen https://github.com/NetronomeMoongen Thanks for the pointers. Actually, I've already been able to send packets from pktgen using the similar setup, I've mentioned before.

But now I want to send using MoonGen , as I need it to resemble a DDOS attack. I've been able to configure one Vhost device in the dpdk-config file like @sarsanaee https://github.com/sarsanaee mentioned. It's getting recognized as Device 0. But I need to configure another vhost device too as Device 1 so that I can send packets.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/253#issuecomment-613893659, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AEX76NI7B7UE7W7Q7BI5SYTRMVVEPANCNFSM4IKCLIMA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/253#issuecomment-613900734, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3ZZPPUYU4YAHCO4K4GEEDRMVWY5ANCNFSM4IKCLIMA .

[sarsanaee]

I find this issue pretty misleading although it is created by myself since I eventually realized I had some misunderstandings regarding DPDK. But still, the responses helped me figure that out.

[immadhavv]

@sarsanaee Thanks! I'm able to initialize multiple virtual ports now. The issue was that I was passing the mac address argument too previously which worked with pktgen but it was a problem here. Also,it was only because of this issue I was able to get started with this. Thanks for that actually.

DPDKConfig { cores={0,3,4}, cli = { "--file-prefix", "moongen", "--socket-mem", "512", --"--rxq","1", --"--txq","1", "--vdev","net_virtio_user1,path=/var/run/openvswitch/vhost-user1,queues=1", "--vdev","net_virtio_user2,path=/var/run/openvswitch/vhost-user2,queues=1", "--log-level","8", } } I've turned the disbleoffloads option to true in device.lua file too. But now, the compiler results in this. says I've requested for 3 rx and tx queues while my devide has only 1 rx and tx queue..but I didn't really ask for 3. I'm trying to pass the rxq and txq arguments here but it won't get accepted(says invalid arguments). It's supposed to be set as q1 by default anyway. `EAL: Virtual area found at 0x7f8505400000 (size = 0xb9000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f8505000000 (size = 0x200000) EAL: Requesting 256 pages of size 2MB from socket 0 EAL: TSC frequency is ~2397223 KHz EAL: Master lcore 0 is ready (tid=ead10940;cpuset=[0]) EAL: lcore 6 is ready (tid=c91fd700;cpuset=[0]) EAL: lcore 7 is ready (tid=c89fc700;cpuset=[0]) EAL: lcore 5 is ready (tid=c99fe700;cpuset=[0]) EAL: lcore 8 is ready (tid=c33ff700;cpuset=[0]) EAL: lcore 9 is ready (tid=c2bfe700;cpuset=[0]) EAL: lcore 10 is ready (tid=c23fd700;cpuset=[0]) EAL: lcore 11 is ready (tid=c1bfc700;cpuset=[0]) EAL: lcore 4 is ready (tid=ca1ff700;cpuset=[4]) EAL: lcore 3 is ready (tid=e7510700;cpuset=[3]) EAL: lcore 2 is ready (tid=e7d11700;cpuset=[2]) EAL: lcore 1 is ready (tid=e8512700;cpuset=[1]) EAL: lcore 12 is ready (tid=c13fb700;cpuset=[0]) EAL: PCI device 0000:00:03.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100e net_e1000_em EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:00:08.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100e net_e1000_em EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:00:09.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100e net_e1000_em EAL: Not managed by a supported kernel driver, skipped EAL: Search driver net_virtio_user1 to probe device net_virtio_user1 EAL: Search driver net_virtio_user2 to probe device net_virtio_user2 [INFO] Found 2 usable devices: Device 0: 8A:E9:DD:7B:A3:89 (unknown NIC (PCI ID 7f85:0)) Device 1: 66:3B:CF:5A:FC:AC (unknown NIC (PCI ID 7830:7030)) [INFO] Flow load-latency => 0x1 [FATAL] Lua error in task master /home/MoonGen/build/../libmoon/lua/device.lua:120: device supports only 1 tx queues, requested 3 Stack Traceback

(2) Lua method 'fatal' at file '/home/MoonGen/build/../libmoon/lua/log.lua:129' Local variables: self = table: 0x40ad6c00 {DEBUG:0, fatal:function: 0x40ad6d30, writeToLog:function: 0x40ad6d10, INFO:1 (more...)} str = string: "device supports only 1 tx queues, requested 3" (3) Lua field 'config' at file '/home/MoonGen/build/../libmoon/lua/device.lua:120' Local variables: args = table: 0x40ee86d8 {bufSize:2048, rxQueues:1, txDescs:1024, txQueues:3, rssQueues:0, numBufs:2047 (more...)} driverInfo = table: 0x40eeb3e0 {} (4) Lua method 'configure' at file './interface/devmgr.lua:40' Local variables: self = table: 0x40ee64e8 {0:table: 0x40ee7b40, 1:table: 0x40ee7c50} (for generator) = C function: next (for state) = table: 0x40ee64e8 {0:table: 0x40ee7b40, 1:table: 0x40ee7c50} (for control) = number: nan i = number: 0 v = table: 0x40ee7b40 {rxq:1, rxqi:0, txq:3, txqi:0, rsqi:0, rsq:0} txq = number: 3 rxq = number: 1 (5) Lua function 'master' at file './interface/init.lua:66' (best guess) Local variables: args = table: 0x419e35a0 {output:., config:flows, start:true, flows:table: 0x419e3f30} devices = table: 0x40ee64e8 {0:table: 0x40ee7b40, 1:table: 0x40ee7c50} flows = table: 0x40ee6598 {1:table: 0x41daf388} (6) global C function 'xpcall' (7) Lua upvalue 'master' at file '/home/MoonGen/build/../libmoon/lua/main.lua:96' Local variables: _ = string: "./build/MoonGen" file = string: "./interface/init.lua" args = table: 0x41855948 {1:start, 2:load-latency:0:1:rate=10Mp/s} cfgFile = string: "libmoon/dpdk-conf.lua" ok = boolean: true parsedArgs = table: 0x419e3538 {1:table: 0x419e35a0} (8) Lua function 'main' at file '/home/MoonGen/build/../libmoon/lua/main.lua:146' (best guess) Local variables: task = string: "master" /home/MoonGen/build/../libmoon/lua/device.lua:120: device supports only 1 tx queues, requested 3`

[immadhavv]

@sarsanaee @NetronomeMoongen @emmericp Could someone take a look at the above issue? I'd like to know how to add the number of rx queues and tx queues as parameters in the dpdk-conf.lua file. I went through the list of EAL parameters but couldn't find how to. It's supposed to be defaulted to 1, but the compiler says I have requested for 3 queues, when I didn't. :/ I tried to manually change the code in device.lua also to set the args.rxQueues and args.txQueues to 1, but it throws another error saying "device is configured with rx queues 0 to 0, tried to get rx queue number 1"

Can someone help me with this and maybe explain how to use the dpdk-conf.lua file to add parameters(a list of all parameters available especially for vdev would help..not just the list of EAL parameters in the dpdk.org)

[sarsanaee]

Did you get it going?