search

lukaszlach / docker-tc

:whale: :vertical_traffic_light: Docker Traffic Control - network rate limiting, emulating delays, losses, duplicates, corrupts and reorders of network packets using only container labels or a command-line interface.
677 stars 40 forks source link

Bandwith limit not working properly #9

Open xoancosmed opened 4 years ago

xoancosmed commented 4 years ago

Bandwitch limit is not working as expected, the expirimented limit is much higher than the value I setted via the label. Here some examples:

$ docker network create test7-net
f91ccaaf402391584ba6afb17dfd1260d91df04f24b76df272c62520a1c520f1
$ docker run -it \
    --net test7-net \
    --label "com.docker-tc.enabled=1" \
    --label "com.docker-tc.limit=256kbps" \
    --label "com.docker-tc.delay=100ms" \
    --label "com.docker-tc.loss=10%" \
    --label "com.docker-tc.duplicate=5%" \
    --label "com.docker-tc.corrupt=1%" \
    mlabbe/iperf3 \
    iperf3 -c 172.20.35.249
Connecting to host 172.20.35.249, port 5201
[  5] local 172.25.0.2 port 49744 connected to 172.20.35.249 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.10 GBytes  9.44 Gbits/sec   12   3.02 MBytes
[  5]   1.00-2.00   sec   806 MBytes  6.76 Gbits/sec    1   3.02 MBytes
[  5]   2.00-3.00   sec  28.8 MBytes   241 Mbits/sec    0   3.02 MBytes
[  5]   3.00-4.00   sec  28.8 MBytes   241 Mbits/sec    0   3.02 MBytes
[  5]   4.00-5.00   sec  30.0 MBytes   252 Mbits/sec    0   3.02 MBytes
[  5]   5.00-6.00   sec  27.5 MBytes   231 Mbits/sec    0   3.02 MBytes
[  5]   6.00-7.00   sec  27.5 MBytes   231 Mbits/sec    0   3.02 MBytes
[  5]   7.00-8.00   sec  28.8 MBytes   241 Mbits/sec    0   3.02 MBytes
[  5]   8.00-9.00   sec  30.0 MBytes   252 Mbits/sec    0   3.02 MBytes
[  5]   9.00-10.00  sec  28.8 MBytes   241 Mbits/sec    0   3.02 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.11 GBytes  1.81 Gbits/sec   13             sender
[  5]   0.00-10.04  sec  2.11 GBytes  1.80 Gbits/sec                  receiver

Do you know where is the problem? I created a new network for each container, I don't know if that's right. I'm using the docker compose YAML to run docker-tc. I'm using Ubuntu Server 18.04 and Docker 19.03.8.

Thanks, Xoán

xoancosmed commented 4 years ago

Hi again,

I've been doing more tests and I found that the problem is with the upload tests (the above ones). These are download tests, and it seems that they work fine:

With this tests I conclude that: first, the rule takes about 2.5 seconds to apply (which is not really a problem in my case), and second, that the rate doesn't apply well to the outgoing traffic.

Taking a quick look to the code I see that the command executed is qdisc_tbf "$IF" rate "$LIMIT", so the problem may not be in docker-tc. Probably one parameter or something is missing, I'm a noobie in terms of tc, but I'll try to investigate it.

spacelatte commented 4 years ago

Hello, @xoancosmed

I saw your benchmarks and even tho I don't have advanced information about tc I noticed 2 things

  1. With iperf you are benchmarking TCP speed while tc sets bandwidth in at most in IP (3rd) Layer and it might be lower (2nd) layer. (TCP and UDP runs on the 4th layer)
  2. In your first post, you had 10% loss, 5% duplicate and 1% corrupt, because of that speeds are lower than upload tests.

I have no idea why you are seeing 2.5 seconds of high bandwidth, it might because of how qdisc (queue) handles them. It might be TCP buffering. There is also a burst option which might be helpful for that kickstart issue.

My suggestion is to either use UDP or raw IP for calculating raw bandwidth.

Ref: https://www.tldp.org/HOWTO/html_single/Traffic-Control-HOWTO/#o-packets

xoancosmed commented 4 years ago

Hi @pvtmert

Thanks for your reply. What you said makes sense ... I'll try it with IPERF in UDP mode. However, the biggest problem for me is that 2.5 seconds where it seems that there is no limit. I have to investigate more about this ...

fischercer commented 4 years ago

Hi @xoancosmed as I stumbled on this issue. The reason for not working in both directions is that tc only works for egress shaping. If you want to use it in both directions you have to apply it on both sides in egress (traffic leaving a container). You could add a limit to the server, which acts in download direction and add a limit to a client container that acts in upload direction. If you do not use a client container you have to apply shaping on the interface the client uses, this however will be applied to all clients connected to this interface.

The reason for the 2,5 sec delay may be that the container is checking the docker api for changes, this takes some time until the rules are applied. It might be better to test against an already running container.

I hope this helps