search

facebookarchive / augmented-traffic-control

Augmented Traffic Control: A tool to simulate network conditions
https://facebook.github.io/augmented-traffic-control
Other
4.33k stars 600 forks source link

Any shaping settings give inconsistent results #341

Open tachu opened 6 years ago

tachu commented 6 years ago

I'm running atcd (0.1.6) on ubuntu 16.04 I've setup the server as a gateway using both nic. One nick is connected to my LAN on 10.24.0.0/24 network seconds nic is initialized on vlan 255 and ip 192.168.128.0/24 Machine is setup as a gateway for clients with dhcp provided. all clients connect to network via wireless on save vlan.

I've run several tests with atcd disabled and speed tests are consistent at around 50mbps down and 30mbps up. enabling atcd for a device drops the speed way below the settings on the profile.

Example is

Cable profile 1000kbps up 6000 down. speed test reports max of 3000 down and 740 up. I've tried increasing the numbers to 12000 down and 6000 up and reports still shows same bandwidth result.

Removing shaping for that device and retesting reports again 50mbps down and 30mbps up.

I notice dmesg reporting [75883.030421] htb: netem qdisc 8004: is non-work-conserving? [75883.089821] htb: netem qdisc 8003: is non-work-conserving? [76228.413021] htb: netem qdisc 8006: is non-work-conserving? [76228.462346] htb: netem qdisc 8005: is non-work-conserving?

HEre is dump info

root@Slacker:~/augmented-traffic-control/utils# ./dump_system_info.sh

######### uname -a #########

Linux Slacker 4.4.0-96-generic #119-Ubuntu SMP Tue Sep 12 14:59:54 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

######### cat /etc/os-release #########

NAME="Ubuntu" VERSION="16.04.3 LTS (Xenial Xerus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 16.04.3 LTS" VERSION_ID="16.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" VERSION_CODENAME=xenial UBUNTU_CODENAME=xenial

######### python -V #########

Python 2.7.12

######### pip freeze #########

adium-theme-ubuntu==0.3.4 atc-thrift==0.1.6 atcd==0.1.6 daemonize==2.4.7 Django==1.11.6 django-atc-api==0.1.6 django-atc-demo-ui==0.1.6 django-atc-profile-storage==0.1.6 django-bootstrap-themes==3.3.6 django-static-jquery==1.11.1 djangorestframework==3.6.4 futures==3.1.1 pyotp==1.4.1 pyroute2==0.3.3 pytz==2017.2 six==1.11.0 sparts==0.7.1 thrift==0.10.0 unity-lens-photos==1.0 You are using pip version 8.1.1, however version 9.0.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

######### ip r #########

default via 10.24.4.1 dev eno1 onlink 10.24.4.0/22 dev eno1 proto kernel scope link src 10.24.4.99 169.254.0.0/16 dev eno2.255 scope link metric 1000 192.168.128.0/24 dev eno2.255 proto kernel scope link src 192.168.128.1

######### iptables-save #########

Generated by iptables-save v1.6.0 on Fri Oct 6 14:47:37 2017

*filter :INPUT ACCEPT [406130:550661434] :FORWARD ACCEPT [452698:675838389] :OUTPUT ACCEPT [192325:11860521] -A FORWARD -i eno2.255 -o eno1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eno2.255 -o eno1 -j ACCEPT COMMIT

Completed on Fri Oct 6 14:47:37 2017

Generated by iptables-save v1.6.0 on Fri Oct 6 14:47:37 2017

*mangle :PREROUTING ACCEPT [22251:11227350] :INPUT ACCEPT [5672:879536] :FORWARD ACCEPT [12067:9571076] :OUTPUT ACCEPT [921:88087] :POSTROUTING ACCEPT [12988:9659163] -A FORWARD -d 192.168.128.106/32 -i eno1 -j MARK --set-xmark 0x2/0xffffffff -A FORWARD -s 192.168.128.106/32 -i eno2.255 -j MARK --set-xmark 0x2/0xffffffff COMMIT

Completed on Fri Oct 6 14:47:37 2017

Generated by iptables-save v1.6.0 on Fri Oct 6 14:47:37 2017

*nat :PREROUTING ACCEPT [24127:3795964] :INPUT ACCEPT [11488:1857333] :OUTPUT ACCEPT [207:12727] :POSTROUTING ACCEPT [187:11286] -A POSTROUTING -o eno1 -j MASQUERADE COMMIT

Completed on Fri Oct 6 14:47:37 2017

######### ip r #########

default via 10.24.4.1 dev eno1 onlink 10.24.4.0/22 dev eno1 proto kernel scope link src 10.24.4.99 169.254.0.0/16 dev eno2.255 scope link metric 1000 192.168.128.0/24 dev eno2.255 proto kernel scope link src 192.168.128.1

######### ip a #########

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000 inet 10.24.4.99/22 brd 10.24.7.255 scope global eno1 valid_lft forever preferred_lft forever valid_lft forever preferred_lft forever 3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 valid_lft forever preferred_lft forever 4: eno2.255@eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000 inet 192.168.128.1/24 brd 192.168.128.255 scope global eno2.255 valid_lft forever preferred_lft forever valid_lft forever preferred_lft forever

######### tc qdisc show dev eno1 #########

qdisc htb 1: root refcnt 6 r2q 10 default 0 direct_packets_stat 94727 direct_qlen 1000 qdisc netem 8005: parent 1:2 limit 1000 delay 2.0ms

######### tc class show dev eno1 #########

class htb 1:2 root leaf 8005: prio 0 rate 5Mbit ceil 5Mbit burst 1600b cburst 1600b

######### tc filter show dev eno1 #########

filter parent 1: protocol ip pref 1 fw filter parent 1: protocol ip pref 1 fw handle 0x2 classid 1:2 police 0x5 rate 5Mbit burst 12000b mtu 2Kb action drop overhead 0b ref 1 bind 1

######### tc qdisc show dev eno2.255 #########

qdisc htb 1: root refcnt 2 r2q 10 default 0 direct_packets_stat 144355 direct_qlen 1000 qdisc netem 8006: parent 1:2 limit 1000 delay 2.0ms

######### tc class show dev eno2.255 #########

class htb 1:2 root leaf 8006: prio 0 rate 12Mbit ceil 12Mbit burst 1597b cburst 1597b

######### tc filter show dev eno2.255 #########

filter parent 1: protocol ip pref 1 fw filter parent 1: protocol ip pref 1 fw handle 0x2 classid 1:2 police 0x6 rate 12Mbit burst 12000b mtu 2Kb action drop overhead 0b ref 1 bind 1

root@Slacker:~/augmented-traffic-control/utils#

Here is after removing shapping

root@Slacker:~/augmented-traffic-control/utils# ./dump_system_info.sh

######### uname -a #########

Linux Slacker 4.4.0-96-generic #119-Ubuntu SMP Tue Sep 12 14:59:54 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

######### cat /etc/os-release #########

NAME="Ubuntu" VERSION="16.04.3 LTS (Xenial Xerus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 16.04.3 LTS" VERSION_ID="16.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" VERSION_CODENAME=xenial UBUNTU_CODENAME=xenial

######### python -V #########

Python 2.7.12

######### pip freeze #########

adium-theme-ubuntu==0.3.4 atc-thrift==0.1.6 atcd==0.1.6 daemonize==2.4.7 Django==1.11.6 django-atc-api==0.1.6 django-atc-demo-ui==0.1.6 django-atc-profile-storage==0.1.6 django-bootstrap-themes==3.3.6 django-static-jquery==1.11.1 djangorestframework==3.6.4 futures==3.1.1 pyotp==1.4.1 pyroute2==0.3.3 pytz==2017.2 six==1.11.0 sparts==0.7.1 thrift==0.10.0 unity-lens-photos==1.0 You are using pip version 8.1.1, however version 9.0.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

######### ip r #########

default via 10.24.4.1 dev eno1 onlink 10.24.4.0/22 dev eno1 proto kernel scope link src 10.24.4.99 169.254.0.0/16 dev eno2.255 scope link metric 1000 192.168.128.0/24 dev eno2.255 proto kernel scope link src 192.168.128.1

######### iptables-save #########

Generated by iptables-save v1.6.0 on Fri Oct 6 14:49:13 2017

*filter :INPUT ACCEPT [407393:550853438] :FORWARD ACCEPT [452736:675851292] :OUTPUT ACCEPT [192893:11910114] -A FORWARD -i eno2.255 -o eno1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eno2.255 -o eno1 -j ACCEPT COMMIT

Completed on Fri Oct 6 14:49:13 2017

Generated by iptables-save v1.6.0 on Fri Oct 6 14:49:13 2017

*mangle :PREROUTING ACCEPT [1559:247082] :INPUT ACCEPT [1036:147283] :FORWARD ACCEPT [24:2711] :OUTPUT ACCEPT [514:44028] :POSTROUTING ACCEPT [538:46739] COMMIT

Completed on Fri Oct 6 14:49:13 2017

Generated by iptables-save v1.6.0 on Fri Oct 6 14:49:13 2017

*nat :PREROUTING ACCEPT [24678:3879108] :INPUT ACCEPT [11779:1899861] :OUTPUT ACCEPT [242:14827] :POSTROUTING ACCEPT [222:13386] -A POSTROUTING -o eno1 -j MASQUERADE COMMIT

Completed on Fri Oct 6 14:49:13 2017

######### ip r #########

default via 10.24.4.1 dev eno1 onlink 10.24.4.0/22 dev eno1 proto kernel scope link src 10.24.4.99 169.254.0.0/16 dev eno2.255 scope link metric 1000 192.168.128.0/24 dev eno2.255 proto kernel scope link src 192.168.128.1

######### ip a #########

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000 inet 10.24.4.99/22 brd 10.24.7.255 scope global eno1 valid_lft forever preferred_lft forever valid_lft forever preferred_lft forever 3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 valid_lft forever preferred_lft forever 4: eno2.255@eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000 inet 192.168.128.1/24 brd 192.168.128.255 scope global eno2.255 valid_lft forever preferred_lft forever valid_lft forever preferred_lft forever

######### tc qdisc show dev eno1 #########

qdisc htb 1: root refcnt 6 r2q 10 default 0 direct_packets_stat 94823 direct_qlen 1000

######### tc class show dev eno1 #########

######### tc filter show dev eno1 #########

######### tc qdisc show dev eno2.255 #########

qdisc htb 1: root refcnt 2 r2q 10 default 0 direct_packets_stat 144506 direct_qlen 1000

######### tc class show dev eno2.255 #########

######### tc filter show dev eno2.255 #########

root@Slacker:~/augmented-traffic-control/utils#

sheenobu commented 6 years ago

Here is a related issue: https://github.com/facebook/augmented-traffic-control/issues/86

The bandwidth looks so low because of packets being dropped and retrasmitted. You can run atcd with the dont-drop-packets flag.

The dropping setting is here: filter parent 1: protocol ip pref 1 fw handle 0x2 classid 1:2 police 0x6 rate 12Mbit burst 12000b mtu 2Kb action drop overhead 0b, specifically the 'action drop overhead'