search

emmericp / MoonGen

MoonGen is a fully scriptable high-speed packet generator built on DPDK and LuaJIT. It can saturate a 10 Gbit/s connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. Multi-core support allows for even higher rates. It also features precise and accurate timestamping and rate control.
MIT License
1.03k stars 234 forks source link

Buffer Allocation issue when running MoonGen for 100G NIC #276

Open sarsanaee opened 4 years ago

sarsanaee commented 4 years ago

Hey,

We are trying to run MoonGen with a 100G Mellonox NIC. However, all examples get a segmentation fault at bufs:alloc line. Is there any idea on this?

Is there something I missed in the setup? I tried to allocate more huge pages or changed the packet size, however, none of these helped!

Thanks, Alireza

emmericp commented 4 years ago

Unfortunately I don't have a 100G Mellanox NIC to test against. But usually the reasopn for seg fault when allocating bufs is that the memory pool ran out of buffers, can you try to increase the size of the mempool?

sarsanaee commented 4 years ago

Hey,

Thanks for the response, by increasing mempool, do you mean I should do something like this? local bufs = mempool:bufArray(64), instead of the default line which is local bufs = mempool:bufArray().

It is pretty weird since the same script works on a 10G Mellanox NIC :)

Thanks in advance, Alireza

emmericp commented 4 years ago

there's an optional first parameter when creating the mempool that specifies the size, default ist 2047:

try this:

memory.createMemPool(65535, function(buf) ... end)

sarsanaee commented 4 years ago

I believe my problem has been resolved. Although the memory allocation wasn't an issue but the OFED version I believe. OFED 4.4 worked properly with the master branch and ubuntu 18.04.

One thing I realized is that OFED 5.1(the newest available OFED) has some issues with DPDK 19.05 branch, and building and linking don't finish.

Thanks, Alireza

sarsanaee commented 4 years ago

I just realized the buffer allocation issue is still there but happens after a few iterations, for instance after a minute or so.

When I use numbers more than 3 local bufs = mempool:bufArray(3) it works for a minute and then crashes with the same seg fault, however, if I don't give input here(local bufs = mempool:bufArray()) it just crashes after 1 iteration.

Thanks, Alireza