Performance of virtionfs

[Peter-JanGootzen]

Currently with XLIO sequential write performance with bs=4k, iodepth=16, numjobs=1 is ~244MB/s, while sequential read only gives ~20MB/s.

• [x] Investigate why this is happening (there are already some flamegraphs ran on commit 7fc9e18)
• [x] Implement a fix

Furthermore blocksizes larger than the page size (4k) don't do anything (i.e. no performance increase).

• [x] Investage why
• [x] Impl fix

[Peter-JanGootzen]

Latency has now also regressed somehow. statfs 50000 runs numactl pinned to right NUMA node and interrupt core XLIO + libnfs -O2 used to be: 82usec NFS: 48usec Host<>DPU: 34usec

Now: Without XLIO: 160usec With XLIO + libnfs -O2: 137usec So we lost 55usec somewhere! Further testing: With XLIO + libnfs -O3: 126usec With XLIO + libnfs -O3 + 2 Virtio queues: 112usec With XLIO + libnfs -O3 + 2 Virtio queues + DPU pinning to core 6,7: 100usec

--enable-latency-measuring Reports that with the last configuration: NFS: 68usec Host<>DPU: 32usec

TODO:

• [x] DPU core pinning = good gains
• [x] Removing -g3 (shouldn't matter) = no change
• [ ] NFS remote side things (the latency measuring results indicate that NFS is the culprit)

[Peter-JanGootzen]

It would be interesting to test whether doing the NFS polling on the Virtio queue polling thread would improve performance. This requires a non-existent libnfs patch.

[Peter-JanGootzen]

• [ ] Try running the NFS remote on a newer server (flex21?)

[Peter-JanGootzen]

TODO disable c-states p-states