Configure upload and download rates separately

[phuedx]

As already noted, it'd be useful to configure upload and download rates separately.

The problem is that you can classify, shape, and police a device's egress traffic but only classify and police it's ingress traffic, for example dropping packets that exceed some rate.

Fortunately, as Sérgio Carvalho explains, one of the policing actions is to redirect traffic to another device, an Intermediate Functional Block (IFB) device. If you shape the egress traffic the IFB, then you're effectively shaping the ingress traffic of the original. This comes at the cost of increased memory usage and delays as traffic flows through the ingress TBF and the HTB, say, attached to the IFB.

[phuedx]

I don't like how the name of the IFB device is hardcoded in 3f23b77. I think there's an opportunity to write a small module that installs the ifb module, requesting N IFBs, and returns their names.

[beenanner]

@phuedx thanks for helping out with this and all the info! I'm new to most of the network shaping and policing magic, so I will try to help out or test where I can.

[phuedx]

I don't like how the name of the IFB device is hardcoded in 3f23b77. I think there's an opportunity to write a small module that installs the ifb module, requesting N IFBs, and returns their names.

See phuedx/node-ifb.

[beenanner]

@phuedx I tested this the other day on my ubuntu machine using my wifi network, but it didn't seem to have any effect on download or upload speeds via speedtest.net?

➜  sltc git:(3f23b77) ✗ sudo tc -s qdisc

qdisc mq 0: dev wlan0 root 
 Sent 15429907 bytes 45727 pkt (dropped 1871, overlimits 0 requeues 9) 
 backlog 0b 0p requeues 9 
qdisc pfifo_fast 0: dev wlan0 parent :1 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 308 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc pfifo_fast 0: dev wlan0 parent :2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc pfifo_fast 0: dev wlan0 parent :3 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 15428411 bytes 45704 pkt (dropped 1871, overlimits 0 requeues 9) 
 backlog 0b 0p requeues 9 
qdisc pfifo_fast 0: dev wlan0 parent :4 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 1188 bytes 22 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 

➜  sltc git:(3f23b77) ✗ bin/sltc.js --device wlan0          

root.INFO: sudo tc qdisc add dev wlan0 handle 1: root htb
root.INFO: sudo tc filter add dev wlan0 protocol ip prio 1 u32 match ip dport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev wlan0 ingress handle ffff:
root.INFO: sudo tc filter add dev wlan0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0
root.INFO: sudo modprobe ifb numifbs=1
root.INFO: sudo ip link set dev ifb0 up
root.INFO: sudo tc qdisc add dev ifb0 handle 1: root htb
root.INFO: sudo tc filter add dev ifb0 protocol ip prio 1 u32 match ip dport 80 0xffff flowid 1:11
root.INFO: sudo tc class add dev wlan0 parent 1: classid 1:11 htb rate 1.6Mbps
root.INFO: sudo tc class add dev ifb0 parent 1: classid 1:11 htb rate 5Mbps
root.INFO: sudo tc qdisc add dev wlan0 parent 1:11 handle 10: netem delay 28ms loss 0%

➜  sltc git:(3f23b77) ✗ sudo tc -s qdisc

qdisc htb 1: dev wlan0 root refcnt 5 r2q 10 default 0 direct_packets_stat 0 direct_qlen 1000
 Sent 66 bytes 1 pkt (dropped 0, overlimits 1 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc netem 10: dev wlan0 parent 1:11 limit 1000 delay 28.0ms
 Sent 66 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc ingress ffff: dev wlan0 parent ffff:fff1 ---------------- 
 Sent 52 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 1: dev ifb0 root refcnt 2 r2q 10 default 0 direct_packets_stat 1 direct_qlen 32
 Sent 66 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 

after running the speedtest.net

➜  sltc git:(3f23b77) ✗ sudo tc -s qdisc     

qdisc htb 1: dev wlan0 root refcnt 5 r2q 10 default 0 direct_packets_stat 45928 direct_qlen 1000
 Sent 15432311 bytes 47681 pkt (dropped 1586, overlimits 9067 requeues 26) 
 backlog 0b 0p requeues 26 
qdisc netem 10: dev wlan0 parent 1:11 limit 1000 delay 28.0ms
 Sent 482748 bytes 1753 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc ingress ffff: dev wlan0 parent ffff:fff1 ---------------- 
 Sent 120487832 bytes 88669 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 1: dev ifb0 root refcnt 2 r2q 10 default 0 direct_packets_stat 80471 direct_qlen 32
 Sent 122155328 bytes 88668 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 

[phuedx]

root.INFO: sudo tc filter add dev wlan0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0
root.INFO: sudo modprobe ifb numifbs=1
root.INFO: sudo ip link set dev ifb0 up

Just a thought: the IFB device should probably be up before we redirect all IP traffic to it.

[beenanner]

@phuedx thanks I'll shift the redirect down to see if it help.

[phuedx]

I see GitHub hasn't picked up the (forced) update I just pushed…

[phuedx]

@beenanner I've been testing phuedx/sltc@22d0efb locally with the speed-test module and have seen the following results:

$ speed-test

      Ping   52 ms
  Download   3.8 Mbps
    Upload   0.4 Mbps

$ bin/sltc.js --bandwidth.upload 200kbit --bandwidth.download 2000kbit
root.INFO: sudo modprobe ifb numifbs=1
root.INFO: sudo ip link set dev ifb0 up
root.INFO: sudo tc qdisc add dev ifb0 handle 1: root htb
root.INFO: sudo tc filter add dev ifb0 protocol ip prio 1 u32 match ip sport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev eth0 handle 1: root htb
root.INFO: sudo tc filter add dev eth0 protocol ip prio 1 u32 match ip dport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev eth0 ingress handle ffff:
root.INFO: sudo tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0
root.INFO: sudo tc class add dev eth0 parent 1: classid 1:11 htb rate 200kbit
root.INFO: sudo tc class add dev ifb0 parent 1: classid 1:11 htb rate 2000kbit
root.INFO: sudo tc qdisc add dev eth0 parent 1:11 handle 10: netem delay 28ms loss 0%

$ speed-test

      Ping   114 ms
  Download   1.8 Mbps
    Upload   0.2 Mbps

N.B. that:

• the ping after applying the netem queuing discipline is ~$base_ping + 2 * $latency; and
• I'm configuring the HTB in kbit (Kilobits per second) so that I didn't have to think to hard while verifying its rate as reported by sudo tc -s class show dev {eth0,ifb0}

[beenanner]

@phuedx sweet verified the same on my Ubuntu machine!

➜  sltc git:(master) speed-test                          

      Ping   100 ms
  Download   10 Mbps
    Upload   2 Mbps

➜  sltc git:(master) bin/sltc.js --device wlan0 --bandwidth.upload 200kbit --bandwidth.download 2000kbit
root.INFO: sudo modprobe ifb numifbs=1
root.INFO: sudo ip link set dev ifb0 up
root.INFO: sudo tc qdisc add dev ifb0 handle 1: root htb
root.INFO: sudo tc filter add dev ifb0 protocol ip prio 1 u32 match ip sport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev wlan0 handle 1: root htb
root.INFO: sudo tc filter add dev wlan0 protocol ip prio 1 u32 match ip dport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev wlan0 ingress handle ffff:
root.INFO: sudo tc filter add dev wlan0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0
root.INFO: sudo tc class add dev wlan0 parent 1: classid 1:11 htb rate 200kbit
root.INFO: sudo tc class add dev ifb0 parent 1: classid 1:11 htb rate 2000kbit
root.INFO: sudo tc qdisc add dev wlan0 parent 1:11 handle 10: netem delay 28ms loss 0%

➜  sltc git:(master) speed-test                         

      Ping   150 ms
  Download   1.2 Mbps
    Upload   0.2 Mbps

[beenanner]

@phuedx / @soulgalore I think we are good to merge these changes into master if you guys are cool with it?

➜  sltc git:(master) ✗ speed-test                          

      Ping   115 ms
  Download   10 Mbps
    Upload   4 Mbps

➜  sltc git:(master) ✗ bin/sltc.js --device wlan0          
root.INFO: sudo modprobe ifb numifbs=1
root.INFO: sudo ip link set dev ifb0 up
root.INFO: sudo tc qdisc add dev ifb0 handle 1: root htb
root.INFO: sudo tc filter add dev ifb0 protocol ip prio 1 u32 match ip sport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev wlan0 handle 1: root htb
root.INFO: sudo tc filter add dev wlan0 protocol ip prio 1 u32 match ip dport 80 0xffff flowid 1:11
root.INFO: sudo tc qdisc add dev wlan0 ingress handle ffff:
root.INFO: sudo tc filter add dev wlan0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0
root.INFO: sudo tc class add dev wlan0 parent 1: classid 1:11 htb rate 200kbps
root.INFO: sudo tc class add dev ifb0 parent 1: classid 1:11 htb rate 625kbps
root.INFO: sudo tc qdisc add dev wlan0 parent 1:11 handle 10: netem delay 28ms loss 0%

➜  sltc git:(master) ✗ speed-test                 

      Ping   174 ms
  Download   4.6 Mbps
    Upload   1.6 Mbps

[soulgalore]

@phuedx send a PR when you have time and I can include it in a Docker build and we can test it out in the wild.