search

mtcp-stack / mtcp

mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems
Other
1.98k stars 436 forks source link

Saturate 1gbps link with 100ms delay #235

Open DBLouis opened 5 years ago

DBLouis commented 5 years ago

Can mtcp saturate a 1gbps link with a latency of 100ms with only 1 connection? I tried to run the example programs with the following configurations:

./epserver -p /srv/www/ -f epserver.conf -N 1

#epserver.conf
io = dpdk
num_cores = 1
num_mem_ch = 1
port = dpdk0
max_num_buffers = 1
rcvbuf = 8192
sndbuf = 58366080
tcp_timeout = 30
tcp_timewait = 0
stat_print = dpdk0

./epwget x.x.x.x/100M.iso 1 -N 1 -c 1 -f epwget.conf

#epwget.conf
io = dpdk
num_cores = 1
num_mem_ch = 1
port = dpdk0
rcvbuf = 58366080
sndbuf = 8192
max_concurrency = 1
max_num_buffers = 1
tcp_timeout = 30
tcp_timewait = 0
stat_print = dpdk0

At best I have 0.5mbps

I use the v2.1 release

eunyoung14 commented 5 years ago

Hi, I'm sorry for the late response. We are currently working with CCP team to incorporate various congestion control algorithms into mTCP and fix existing issues on our implementation. They have made pull requests to us (#209, #224), so I'll let you know once I finish testing and merging those PRs.

DBLouis commented 5 years ago

Thank you for your answer. To be clear, it is not possible to get 1gbps with 1 connection and 100ms RTT with the stable branch of mTCP, right? I tried to hard-code buffer sizes and window scaling factor and did not get better results. I do not understand why.

eunyoung14 commented 5 years ago

I actually haven't tested mTCP with such settings, so can't confirm the performance you got. The ongoing issues about the congestion control implementation could be the reason, but not sure it's the root cause because the performance you got is exceptionally low. By the way, is it possible to saturate 1G with single connection/100ms RTT even in the stock Linux stack?

DBLouis commented 5 years ago

With iperf3 under the same conditions I get ~250mbps. And I just realized I had put 0.05gbps but it is actually 0.5mbps.

eunyoung14 commented 5 years ago

I also got the similar throughput with Linux. I'll verify this settings in the future updates.