Xilinx-CNS / tcpdirect

AMD TCPDirect ultra low latency kernel bypass TCP and UDP implementation for AMD Solarflare network adapters, to be used with corresponding versions of Onload®️ at https://github.com/Xilinx-CNS/onload. The stable branch is currently `v8_1`.
22 stars 17 forks source link

ON-16111: use monitor thread in zfsink to allow testing higher rates #50

Closed rdrinkwa-xilinx closed 2 weeks ago

rdrinkwa-xilinx commented 1 month ago

Modified zfsink to use a monitoring thread to output traffic rate every second, as with efsink in Onload.

Tested with high packet rate (>5Mpps) and high throughput (>9Gbps) to confirm output is correctly formatted. Timestamps work as before and are output every packet. Monitoring thread can be disabled using '-q'.

rdrinkwa-xilinx commented 1 month ago

I refactored the code to just pass the resource structure between functions because this felt cleaner to me and was also the approach used for efsink.

Result from testing different options:

[server1:static]$ ./zfsink 224.1.2.3:12345 # two bursts of 100 packets
# pkt-rate  bandwidth(Mbps)       total-pkts
         0                0                 0
         0                0                 0
       100                0               100
         0                0               100
       100                0               200
         0                0               200
^C
[server1:static]$ ./zfsink -m 224.1.2.3:12345 # two bursts of 100 packets
# pkt-rate  bandwidth(Mbps)       total-pkts
         0                0                 0
         0                0                 0
       100                0               100
       100                0               200
         0                0               200
^C
[server1:static]$ ./zfsink -w 224.1.2.3:12345 # two bursts of 100 packets
# pkt-rate  bandwidth(Mbps)       total-pkts
         0                0                 0
         0                0                 0
       100                0               100
       100                0               200
         0                0               200
         0                0               200
^C
[server1:static]$ ./zfsink -r 224.1.2.3:12345 # one burst of 5 packets
# pkt-rate  bandwidth(Mbps)       total-pkts
         0                0                 0
         0                0                 0
Hardware timestamp: 1729093248.149723774
Hardware timestamp: 1729093248.149724426
Hardware timestamp: 1729093248.149724499
Hardware timestamp: 1729093248.149724575
Hardware timestamp: 1729093248.149724651
         5                0                 5
         0                0                 5
         0                0                 5
^C
[server1:static]$ ./zfsink -qr 224.1.2.3:12345 # one burst of 5 packets
Hardware timestamp: 1729093258.533324537
Hardware timestamp: 1729093258.533325183
Hardware timestamp: 1729093258.533325258
Hardware timestamp: 1729093258.533325333
Hardware timestamp: 1729093258.533325410
^C
[server1:static]$ ./zfsink 224.1.2.3:12345 # efsend using 1 byte payload no throttling
# pkt-rate  bandwidth(Mbps)      total-pkts
         0                0                0
         0                0                0
         0                0                0
         0                0                0
   3534625               28          3534625
   5248144               41          8782769
   5247216               41         14029985
   5248816               41         19278801
   5248128               41         24526929
^C
[server1:static]$
pemberso-xilinx commented 1 month ago

Thanks for the output, looks fine.

I refactored the code to just pass the resource structure between functions because this felt cleaner to me and was also the approach used for efsink.

efsink needs to manage packet buffers so there is more need to share state. I'd rather have a minimal code change here - e.g. suggest just adding a couple of global stats counters and then most of the changes to existing code won't be needed. Code can always be refactored later if more data needs to be shared but doesn't seem necessary here.