search

smoltcp-rs / smoltcp

a smol tcp/ip stack
BSD Zero Clause License
3.75k stars 421 forks source link

Optimizing smoltcp->smoltcp (no operating system) throughput with loopback example #884

Open cameronelliott opened 9 months ago

cameronelliott commented 9 months ago

Thanks to the creators, and maintainers of smoltcp, it is incredible.

I was wondering what the throughput of smoltcp is without hitting the operating system.

So, I hacked up the loopback example, see attachment.

And I seem to be getting about 5GB/s or 42Gb/s on an Intel 12400 CPU

elapsed: Duration { micros: 1874114 }
ntx: 10000084022
nrx: 10000018526
throughput: 5.3 GBps
throughput: 42.7 Gbps

I am wondering if that seems about right for TCP throughput, when going Smol->smol_loopback->Smol without hitting an operating system devices, ie, tap, tun, etc.

If you have any suggestions on improving the performance, please let me know.

Attached is the modified loopback example. (I got rid of pcapwriter, etc) (it is renamed from .rs to .txt as github doesn't like .rs attachments)

loopback-bench.txt

cameronelliott commented 9 months ago

Interestingly, I get nearly 100Gb/sec with iperf on that same system using the OS loopback interface, so it would seem I might be able to improve my speeds with smoltcp somehow. (I could try two threads for example)

iperf client

c@intel12400 ~/smoltcp (main)> iperf -c 127.0.0.1
------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[  1] local 127.0.0.1 port 33038 connected with 127.0.0.1 port 5001 (icwnd/mss/irtt=319/32741/9)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-10.0159 sec   115 GBytes  98.7 Gbits/sec

iperf server

c@intel12400 ~> iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
[  1] local 127.0.0.1 port 5001 connected with 127.0.0.1 port 33038 (icwnd/mss/irtt=320/32768/7)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-10.0004 sec   115 GBytes  98.8 Gbits/sec
thvdveld commented 8 months ago

It would be nice if we could also do it for IPv6!