F-Stack latency is worse than kernel

[vitalif]

Hi

I've made a simple ping-pong application (see here - https://github.com/vitalif/f-stack) and tested in on two directly connected PCs with 10G cards (Intel X520-DA2 and Solarflare SFN5122F). My test application isn't correct for messages longer than the MTU, because the server immediately sends each packet back instead of waiting for the whole message, but for 1500 bytes it's fine. CPU powersaving was off during test (cpupower idle-set -D 0 && cpupower frequency-set -g performance). F-Stack's idle_sleep and pkt_tx_delay were 0.

I only got 26us latency with 1500 byte messages which seems quite a lot for a kernel-bypass 10G network. In fact, I get better latency with normal Linux TCP stack! sockperf sr --tcp and sockperf pp --tcp -m 1500 -i SERVER_IP -t 5 gives me 17.8us latency with the kernel TCP stack.

Is it fine that F-Stack is slower than the kernel?) How to achieve low latency with F-Stack?

[vitalif]

For those who are curious: sockperf prints half of the latency. :-) so f-stack vs kernel was 26us vs 36us in my test which seems fine. I still don't use it for my project because it's not easy to use it because multiple processes have to run on the same server and NIC in my case. So I need either to rework the architecture or to invent some IPC mechanisms, which, in turn, will increase the latency back. I may return to testing it some day, but not now...