env: FreeBSD 14.0 current ( just 1 week old on this day )
system has 8GB RAM with ZFS
card IXGBE 550 T2 with 8 hw queues
processor: i5 6500 , 4 cores ( this processor has no hyper threading)
i am new to github and please pardon me for any mistakes. Excellent project and thanks for putting it into open-source domain.
I have some good experience at C programming and is facing an abnormal situation. I am interested in only packet capture ( no TX required).
my app cannot guarantee that it will be be able to poll the fd as fast as possible. ( due to unavoidable reasons, latencies for POLL can be upto 10 ms/milliseconds between successive polls)
also note that i am requesting 16384 extra bufs per ring to implement ZC, in Arg3
default netmap comes with rxd size 2048,
if i change the rxds using the setting in /boot/loader.conf
dev.ix.0.iflib.override_nrxds="4096"
and start my app i get the following messages
kernel: 754.834915 [ 852] iflib_netmap_config txr 4 rxr 4 txd 2048 rxd 4096 rbufsz 2048
kernel: 754.835050 [1026] netmap_obj_malloc netmap_ring request size 65792 too large
kernel: 754.835061 [2001] netmap_mem2_rings_create Cannot allocate RX_ring
since default ring in freeBSD is around 38000
sysctl dev.netmap.ring_size=69000
and restarting application, it works fine
Till above everything is fine.
However, i have patched the ixgbe driver in freebsd to allow max nrxds from 4096 to 32768
after compiling kernel and all, i set this in /boot/loader.conf to have 8192 descriptors
dev.ix.0.iflib.override_nrxds="8192"
at this point, all networking etc ( non netmap things ) are working fine.
now when i start the my app the following kernel messages comes,
at this point, netmap tells me ring size is less, just like the above case, so i set it like this
sysctl dev.netmap.ring_size=139000
however, now when i start the app, the get the following messages.
kernel: 267.695175 [1347] netmap_config_obj_allocator aligning object by 32 bytes
kernel: 267.695187 [1352] netmap_config_obj_allocator requested objsize 140032 out of range [256, 131072]
after patching netmap and increasing max size, everything works great.
so my question is can it be controlled via sysctl or should i patch this into source code i maintain.
also note that i updated the max size of object in object pool.
Can you please provide a sysctl for freebsd ( or other platforms ) to control max_size of ring and max size of object in your allocator pools or whether it could be linked to existing sysctls ( this way we do not need new sysctls and existing sysctl will get the job done.)
for eg.
if dev.netmap.ring_size is greater than max size of ring pool, then automatically increase the max limit of pool.
same can be done with object pool,
if ____ is greater than max size of ring pool, then automatically increase the max object size of pool
env: FreeBSD 14.0 current ( just 1 week old on this day ) system has 8GB RAM with ZFS card IXGBE 550 T2 with 8 hw queues processor: i5 6500 , 4 cores ( this processor has no hyper threading)
i am new to github and please pardon me for any mistakes. Excellent project and thanks for putting it into open-source domain. I have some good experience at C programming and is facing an abnormal situation. I am interested in only packet capture ( no TX required). my app cannot guarantee that it will be be able to poll the fd as fast as possible. ( due to unavoidable reasons, latencies for POLL can be upto 10 ms/milliseconds between successive polls)
also note that i am requesting 16384 extra bufs per ring to implement ZC, in Arg3
default netmap comes with rxd size 2048, if i change the rxds using the setting in /boot/loader.conf
dev.ix.0.iflib.override_nrxds="4096"and start my app i get the following messages kernel: 754.834915 [ 852] iflib_netmap_config txr 4 rxr 4 txd 2048 rxd 4096 rbufsz 2048 kernel: 754.835050 [1026] netmap_obj_malloc netmap_ring request size 65792 too large kernel: 754.835061 [2001] netmap_mem2_rings_create Cannot allocate RX_ring
since default ring in freeBSD is around 38000
sysctl dev.netmap.ring_size=69000 and restarting application, it works fine
Till above everything is fine. However, i have patched the ixgbe driver in freebsd to allow max nrxds from 4096 to 32768 after compiling kernel and all, i set this in /boot/loader.conf to have 8192 descriptors
dev.ix.0.iflib.override_nrxds="8192"at this point, all networking etc ( non netmap things ) are working fine. now when i start the my app the following kernel messages comes,
kernel: 076.863860 [ 852] iflib_netmap_config txr 4 rxr 4 txd 2048 rxd 8192 rbufsz 2048 kernel: 076.864016 [1026] netmap_obj_malloc netmap_ring request size 131328 too large kernel: 076.864027 [2001] netmap_mem2_rings_create Cannot allocate RX_ring
at this point, netmap tells me ring size is less, just like the above case, so i set it like this sysctl dev.netmap.ring_size=139000
however, now when i start the app, the get the following messages. kernel: 267.695175 [1347] netmap_config_obj_allocator aligning object by 32 bytes kernel: 267.695187 [1352] netmap_config_obj_allocator requested objsize 140032 out of range [256, 131072]
i have checked netmap source code, and figured out the following limits https://github.com/luigirizzo/netmap/blob/bd5813a69bfb14e8f5d9979e7858aa6935dccf1f/sys/dev/netmap/netmap_mem2.c#L551
which states that 32*4096 = 131072
after patching netmap and increasing max size, everything works great. so my question is can it be controlled via sysctl or should i patch this into source code i maintain.
also note that i updated the max size of object in object pool. Can you please provide a sysctl for freebsd ( or other platforms ) to control max_size of ring and max size of object in your allocator pools or whether it could be linked to existing sysctls ( this way we do not need new sysctls and existing sysctl will get the job done.) for eg. if dev.netmap.ring_size is greater than max size of ring pool, then automatically increase the max limit of pool.
same can be done with object pool, if ____ is greater than max size of ring pool, then automatically increase the max object size of pool
Thank you.