search

dtaht / sch_cake

Out of tree build for the new cake qdisc
100 stars 35 forks source link

Getting Cake working on DD-WRT the right way #151

Closed brbergami closed 2 years ago

brbergami commented 3 years ago

Hi there, I'm struggling with dd-wrt to get cake work as intended.

My setup is:

fiber cable -> Technicolor CGA4233TCH3 -> RJ45 -> Linksys WRT3200ACM -> RJ45 client

ISP Fibertel (Fiber), 300/30 (332/35 with "burst") with Docsis 3.1 (afaik). WRT has DD-WRT build r47000 at this moment. Disabled QoS from settings to avoid HTB being used (dd-wrt HAS cake/pie implementation, but looking at changelog, this implementation is kinda clunky, doesn't disable HTB, just besteffort, etc...) and created a startup script based on bufferbloat.net example. Checked tc-cake manual, tuned some stuff. My goal is to avoid latency on games and videocalls (like Meet).

Actual script:

modprobe sch_cake
modprobe ifb
modprobe act_mirred

tc qdisc add dev eth0 root cake docsis diffserv4 dual-srchost ack-filter nat bandwidth 24mbit

ip link add name ifb0 type ifb
tc qdisc del dev eth0 ingress
tc qdisc add dev eth0 handle ffff: ingress
tc qdisc del dev ifb0 root
tc qdisc add dev ifb0 root cake docsis besteffort dual-dsthost wash nat bandwidth 264mbit
ip link set ifb0 up
tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb0

I want to know what are my mistakes in this script. Still getting heavy rubberbanding on games and calls degrade untill video freezes every 30 sec. I don't know if the line of ingress goes before egress or viceversa. Not sure if docsis keyword can be used in the network edge (Linksys) or I should be using ethernet keyword for a safer overhead / mpu since the Linksys communicates with the gateway using the actual ethernet cable. bandwidth is pretty conservative considering the speed boost my ISP offers.

Honestly, copy/pasting code from random people in the internet trying to make it work as me leads me nowhere and I want to understand this the right way.

Cheers!

tohojo commented 3 years ago

Bruno Bergami @.***> writes:

Hi there, I'm struggling with dd-wrt to get cake work as intended.

My setup is:

fiber cable -> Technicolor CGA4233TCH3 -> RJ45 -> Linksys WRT3200ACM -> RJ45 client

ISP Fibertel (Fiber), 300/30 (332/35 with "burst") with Docsis 3.1 (afaik). WRT has DD-WRT build r47000 at this moment. Disabled QoS from settings to avoid HTB being used (dd-wrt HAS cake/pie implementation, but looking at changelog, this implementation is kinda clunky, doesn't disable HTB, just besteffort, etc...) and created a startup script based on bufferbloat.net example. Checked tc-cake manual, tuned some stuff. My goal is to avoid latency on games and videocalls (like Meet).

Actual script:

modprobe sch_cake
modprobe ifb
modprobe act_mirred

tc qdisc add dev eth0 root cake docsis diffserv4 dual-srchost ack-filter nat bandwidth 24mbit

ip link add name ifb0 type ifb
tc qdisc del dev eth0 ingress
tc qdisc add dev eth0 handle ffff: ingress
tc qdisc del dev ifb0 root
tc qdisc add dev ifb0 root cake docsis besteffort dual-dsthost wash nat bandwidth 264mbit
ip link set ifb0 up
tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb0

I want to know what are my mistakes in this script. Still getting heavy rubberbanding on games and calls degrade untill video freezes every 30 sec. I don't know if the line of ingress goes before egress or viceversa. Not sure if docsis keyword can be used in the network edge (Linksys) or I should be using ethernet keyword for a safer overhead / mpu since the Linksis communicates with the gateway using the actual ethernet cable. bandwidth is pretty conservative considering the speed boost my ISP offers.

Honestly, copy/pasting code from random people in the internet trying to make it work as me leads me nowhere and I want to understand this the right way.

The setup looks OK; did you check if CAKE is handling/dropping packets? The output of 'tc -s qdisc' might be illuminating...

brbergami commented 3 years ago

Sure thing, here:

qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev eth0 root 
 Sent 536530797 bytes 553481 pkt (dropped 0, overlimits 0 requeues 11800) 
 backlog 0b 0p requeues 11800 
qdisc fq_codel 0: dev eth0 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 536530797 bytes 553481 pkt (dropped 0, overlimits 0 requeues 11800) 
 backlog 0b 0p requeues 11800 
  maxpacket 8940 drop_overlimit 0 new_flow_count 6521 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev eth1 root 
 Sent 393652893 bytes 337798 pkt (dropped 0, overlimits 0 requeues 2327) 
 backlog 0b 0p requeues 2327 
qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 393652893 bytes 337798 pkt (dropped 0, overlimits 0 requeues 2327) 
 backlog 0b 0p requeues 2327 
  maxpacket 27252 drop_overlimit 0 new_flow_count 1908 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev wlan0 root 
 Sent 41151074 bytes 226875 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 0: dev wlan0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 41147953 bytes 226854 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 4542 drop_overlimit 0 new_flow_count 158 ecn_mark 0
  new_flows_len 1 old_flows_len 4
qdisc fq_codel 0: dev wlan0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 3121 bytes 21 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev wlan1 root 
 Sent 87999 bytes 687 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 0: dev wlan1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 87999 bytes 687 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8002: dev ifb0 root refcnt 2 bandwidth 264000Kbit besteffort dual-dsthost nat wash no-ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64 no-sce 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
 memory used: 0b of 13200000b
 capacity estimate: 264000Kbit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh     264000Kbit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

So, docsis option is ok despite the setup?

And before you ask, yes, my wan facing interface is eth0 😋 image

chromi commented 3 years ago

On 8 Jul, 2021, at 10:50 pm, Bruno Bergami @.***> wrote:

qdisc mq 0: dev eth0 root

I see your problem. Instead of "tc qdisc add", I recommend "tc qdisc replace", which could simplify your script to:

modprobe sch_cake modprobe ifb modprobe act_mirred

tc qdisc replace dev eth0 root cake docsis diffserv4 dual-srchost ack-filter nat bandwidth 24mbit

ip link add name ifb0 type ifb tc qdisc replace dev eth0 handle ffff: ingress tc qdisc replace dev ifb0 root cake docsis besteffort dual-dsthost wash nat bandwidth 264mbit ip link set ifb0 up tc filter replace dev eth0 parent ffff: matchall action mirred egress redirect dev ifb0

brbergami commented 3 years ago

Does replace, adds the device when there's not a previous one?

chromi commented 3 years ago

On 8 Jul, 2021, at 11:54 pm, Bruno Bergami @.***> wrote:

Does replace, adds the device when there's not a previous one?

It's not adding a device, but a qdisc or filter. But yes, it acts as "del" then "add".

brbergami commented 3 years ago

Ok. Changed. But still looking empy stats?

qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev eth0 root 
 Sent 172332960 bytes 257889 pkt (dropped 0, overlimits 0 requeues 1555) 
 backlog 0b 0p requeues 1555 
qdisc fq_codel 0: dev eth0 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 172332960 bytes 257889 pkt (dropped 0, overlimits 0 requeues 1555) 
 backlog 0b 0p requeues 1555 
  maxpacket 9084 drop_overlimit 0 new_flow_count 15726 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev eth1 root 
 Sent 737231126 bytes 532907 pkt (dropped 0, overlimits 0 requeues 38) 
 backlog 0b 0p requeues 38 
qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 737231126 bytes 532907 pkt (dropped 0, overlimits 0 requeues 38) 
 backlog 0b 0p requeues 38 
  maxpacket 12112 drop_overlimit 0 new_flow_count 12920 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev wlan0 root 
 Sent 2828388 bytes 24472 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 0: dev wlan0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 2826858 bytes 24464 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 3028 drop_overlimit 0 new_flow_count 68 ecn_mark 0
  new_flows_len 1 old_flows_len 6
qdisc fq_codel 0: dev wlan0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 1530 bytes 8 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev wlan1 root 
 Sent 7786838 bytes 7022 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 0: dev wlan1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 7785477 bytes 7010 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 4542 drop_overlimit 0 new_flow_count 35 ecn_mark 0
  new_flows_len 1 old_flows_len 3
qdisc fq_codel 0: dev wlan1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn 
 Sent 1361 bytes 12 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8002: dev ifb4eth0 root refcnt 2 bandwidth 264000Kbit besteffort dual-dsthost nat wash no-ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64 no-sce 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
 memory used: 0b of 13200000b
 capacity estimate: 264000Kbit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh     264000Kbit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
moeller0 commented 3 years ago

Not only are the stats for the cake instance on ifb4eth0 empty, also the whole cake instance for eth0 seems missing...

Regards Sebastian

On Jul 8, 2021, at 23:04, Bruno Bergami @.***> wrote:

Ok. Changed. But still looking empy stats?

qdisc noqueue 0: dev lo root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc mq 0: dev eth0 root Sent 172332960 bytes 257889 pkt (dropped 0, overlimits 0 requeues 1555) backlog 0b 0p requeues 1555r qdisc fq_codel 0: dev eth0 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 172332960 bytes 257889 pkt (dropped 0, overlimits 0 requeues 1555) backlog 0b 0p requeues 1555 maxpacket 9084 drop_overlimit 0 new_flow_count 15726 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc mq 0: dev eth1 root Sent 737231126 bytes 532907 pkt (dropped 0, overlimits 0 requeues 38) backlog 0b 0p requeues 38 qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 737231126 bytes 532907 pkt (dropped 0, overlimits 0 requeues 38) backlog 0b 0p requeues 38 maxpacket 12112 drop_overlimit 0 new_flow_count 12920 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc noqueue 0: dev br0 root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc mq 0: dev wlan0 root Sent 2828388 bytes 24472 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc fq_codel 0: dev wlan0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev wlan0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 2826858 bytes 24464 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 3028 drop_overlimit 0 new_flow_count 68 ecn_mark 0 new_flows_len 1 old_flows_len 6 qdisc fq_codel 0: dev wlan0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev wlan0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 1530 bytes 8 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc mq 0: dev wlan1 root Sent 7786838 bytes 7022 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc fq_codel 0: dev wlan1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev wlan1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 7785477 bytes 7010 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 4542 drop_overlimit 0 new_flow_count 35 ecn_mark 0 new_flows_len 1 old_flows_len 3 qdisc fq_codel 0: dev wlan1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc fq_codel 0: dev wlan1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Sent 1361 bytes 12 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 qdisc cake 8002: dev ifb4eth0 root refcnt 2 bandwidth 264000Kbit besteffort dual-dsthost nat wash no-ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64 no-sce Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 memory used: 0b of 13200000b capacity estimate: 264000Kbit min/max network layer size: 65535 / 0 min/max overhead-adjusted size: 65535 / 0 average network hdr offset: 0

              Tin 0

thresh 264000Kbit target 5.0ms interval 100.0ms pk_delay 0us av_delay 0us sp_delay 0us backlog 0b pkts 0 bytes 0 way_inds 0 way_miss 0 way_cols 0 marks 0 drops 0 ack_drop 0 sp_flows 0 bk_flows 0 un_flows 0 max_len 0

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

chromi commented 3 years ago

On 9 Jul, 2021, at 12:39 am, moeller0 @.***> wrote:

Not only are the stats for the cake instance on ifb4eth0 empty, also the whole cake instance for eth0 seems missing...

Actually, the fact that the one cake instance is on ifb4eth0 is a clue. The script is setting up ifb0 and putting cake on that. So what we're seeing here is actually the vestiges of the default setup, and the script has done nothing for some reason.

How are you running the script? Is it reporting any errors? Are you running it in a context that error messages would be visible?

brbergami commented 3 years ago

I haven't enabled log mode from dd-wrt. I'm not sure it will report something related but worth the shot. I'll try that at home and report here.

And just one thing I changed without comment here: the name from ifb0 to ifb4eth0 since I saw in this repo readme that the name must be linked to the interface that you're trying to shape. But as you can see, before and after you see no stats in both.

Will report sooner here if I get some logs. Ty in advance.

brbergami commented 3 years ago

UPDATE: Enabled syslogd, but in the syslog tab doesn't seems to occur any event related to the startup script, tc, or ip. Any idea?

chromi commented 3 years ago

On 9 Jul, 2021, at 6:17 am, Bruno Bergami @.***> wrote:

UPDATE: Enabled syslogd, but in the syslog tab doesn't seems to occur any event related to the startup script, tc, or ip. Any idea?

I would expect errors to appear on stderr (on the terminal) when you run the script. How are you running the script?

xnoreq commented 3 years ago

Hello @brbergami I see a couple of issues. Commenting on your initial script:

The modprobes, if needed at all, seem fine. On openwrt installing kmod-sched, kmod-sched-{cake,core,connmark...} will automatically load all these modules on boot.

Next, I recommend a cleanup section in your script. I would not use replace:

tc qdisc del dev eth0 root
tc qdisc del dev eth0 ingress
ip link del dev ifb0

This deletes the root qdisc (which implements the packet scheduling for outgoing aka egress on eth0), the ingress policer (for incoming packets), and finally the ifb0 device.

You can discard errors by adding 2> /dev/null at the end of each line, but I wouldn't recommend hiding errors until you understand what is going on.

As a sanity check, you can do a tc qdisc show dev eth0 after this. This should show a single default root qdisc, which could be "noqueue", "pfifo" or "fq_codel" on more modern systems. Don't worry about this, since this default root qdisc will be automatically replaced as soon as you add your own. If you do the same for ifb0 you should get an error from tc since we deleted the interface in the cleanup.

Now, on to the setup, most of what you have in your initial script is fine. You add a cake as root qdisc on eth0, create the ifb0 device, set up the eth0 ingress, add a cake to ifb0 (you should also add the cake parameter "ingress" here), and finally add a filter to eth0 ingress that redirects incoming packets to ifb0.

All of the ifb stuff is a workaround since ingress only allows policing (which doesn't allow us to do elaborate packet scheduling such as with cake). So we redirect incoming packets from eth0 to the ifb0 device, so the packets can go through ifb0's root qdisc. Basically, this allows us to process incoming packets as if they were outgoing (egress) packets, going through another cake qdisc.

After setup check tc qdisc show dev eth0. You should see cake as root qdisc, and the ingress qdisc. Now check if the ingress filter is there: tc filter show dev eth0 parent ffff:. Also check ifb0: tc qdisc show dev ifb0. You should only see cake as root qdisc.

You can add -s to any of these tc commands to show packet/byte statistics for the qdiscs and filters.

--

Finally, a hint for DD-WRT: disable SFE (Shortcut Forwarding Engine) if it is enabled, or any other form of acceleration that could bypass parts of Linux' network stack.

Also, I'm not sure how powerful the hardware in your router is, but cake is slow and typical home routers will struggle, especially at your speeds. You may need to switch from diffserv4 to besteffort and maybe disabling ack-filter and wash helps as well. If you have a working setup just start a fast download and monitor CPU load.

brbergami commented 3 years ago

On 9 Jul, 2021, at 6:17 am, Bruno Bergami @.***> wrote: UPDATE: Enabled syslogd, but in the syslog tab doesn't seems to occur any event related to the startup script, tc, or ip. Any idea? I would expect errors to appear on stderr (on the terminal) when you run the script. How are you running the script? - Jonathan Morton

Using startup script as always. Usually worked. Not this time. image

Hello @brbergami I see a couple of issues. Commenting on your initial script:

The modprobes, if needed at all, seem fine. On openwrt installing kmod-sched, kmod-sched-{cake,core,connmark...} will automatically load all these modules on boot.

Next, I recommend a cleanup section in your script. I would not use replace:

tc qdisc del dev eth0 root
tc qdisc del dev eth0 ingress
ip link del dev ifb0

This deletes the root qdisc (which implements the packet scheduling for outgoing aka egress on eth0), the ingress policer (for incoming packets), and finally the ifb0 device.

You can discard errors by adding 2> /dev/null at the end of each line, but I wouldn't recommend hiding errors until you understand what is going on.

As a sanity check, you can do a tc qdisc show dev eth0 after this. This should show a single default root qdisc, which could be "noqueue", "pfifo" or "fq_codel" on more modern systems. Don't worry about this, since this default root qdisc will be automatically replaced as soon as you add your own. If you do the same for ifb0 you should get an error from tc since we deleted the interface in the cleanup.

Now, on to the setup, most of what you have in your initial script is fine. You add a cake as root qdisc on eth0, create the ifb0 device, set up the eth0 ingress, add a cake to ifb0 (you should also add the cake parameter "ingress" here), and finally add a filter to eth0 ingress that redirects incoming packets to ifb0.

All of the ifb stuff is a workaround since ingress only allows policing (which doesn't allow us to do elaborate packet scheduling such as with cake). So we redirect incoming packets from eth0 to the ifb0 device, so the packets can go through ifb0's root qdisc. Basically, this allows us to process incoming packets as if they were outgoing (egress) packets, going through another cake qdisc.

After setup check tc qdisc show dev eth0. You should see cake as root qdisc, and the ingress qdisc. Now check if the ingress filter is there: tc filter show dev eth0 parent ffff:. Also check ifb0: tc qdisc show dev ifb0. You should only see cake as root qdisc.

You can add -s to any of these tc commands to show packet/byte statistics for the qdiscs and filters.

--

Finally, a hint for DD-WRT: disable SFE (Shortcut Forwarding Engine) if it is enabled, or any other form of acceleration that could bypass parts of Linux' network stack.

Also, I'm not sure how powerful the hardware in your router is, but cake is slow and typical home routers will struggle, especially at your speeds. You may need to switch from diffserv4 to besteffort and maybe disabling ack-filter and wash helps as well. If you have a working setup just start a fast download and monitor CPU load.

Lot of questions here. As far as I follow you, my script should look like this?:

#Cleanup
tc qdisc del dev eth0 root
2> /dev/null
tc qdisc del dev eth0 ingress
2> /dev/null
ip link del dev ifb0
2> /dev/null

#Egress
tc qdisc add dev eth0 root cake egress docsis diffserv4 dual-srchost ack-filter nat bandwidth 24mbit

#Ingress
ip link add name ifb4eth0 type ifb
tc qdisc del dev eth0 ingress
tc qdisc add dev eth0 handle ffff: ingress
tc qdisc del dev ifb4eth0 root
tc qdisc add dev ifb4eth0 root cake ingress docsis besteffort dual-dsthost wash nat bandwidth 264mbit
ip link set ifb4eth0 up
tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb4eth0

After replacing the script to this ^, running the command doesn't looks promising, since I should be getting the cake qdisc in here, isn't it?:

tc qdisc show dev eth0
qdisc mq 0: root
qdisc fq_codel 0: parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc ingress ffff: parent ffff:fff1 ----------------

This router is quite powerful, rarely I see hitting 0.5 and max is 2.0 afaik. It's a 1.8Ghz dual core. Like, damn, cannot handle a couple of processes? It's not like I'm running multiple gigabit WAN in here.

brbergami commented 3 years ago

Also, disabled SFE. It's enabled by default in newer dd-wrt builds. and regarding the preset, the wash and ack-filter, I'll try those once I confirm shaper is taking place at least.

brbergami commented 3 years ago

This is what it looks like if I remove my startup script and enable the default cake implementation by dd-wrt (which I don't want to use, as I mentioned before because uses HTB and a generic implementation)

tc qdisc show dev eth0
qdisc htb 1: root refcnt 9 r2q 10 default 30 direct_packets_stat 0
qdisc cake 30: parent 1:30 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
qdisc cake 100: parent 1:100 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
qdisc cake 10: parent 1:10 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
qdisc cake 40: parent 1:40 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
qdisc cake 20: parent 1:20 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce

And the stats:

tc -s qdisc show dev eth0
qdisc htb 1: root refcnt 9 r2q 10 default 30 direct_packets_stat 0
 Sent 1417930 bytes 9229 pkt (dropped 0, overlimits 0 requeues 6)
 backlog 0b 0p requeues 6
qdisc cake 30: parent 1:30 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
 Sent 1417930 bytes 9229 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 19792b of 4Mb
 capacity estimate: 0bit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       84 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh           0bit
  target          5.0ms
  interval      100.0ms
  pk_delay          3us
  av_delay          2us
  sp_delay          1us
  backlog            0b
  pkts             9229
  bytes         1417930
  way_inds           68
  way_miss         1254
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len         13248
  quantum          1514

qdisc cake 100: parent 1:100 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 0b of 4Mb
 capacity estimate: 0bit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh           0bit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

qdisc cake 10: parent 1:10 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 0b of 4Mb
 capacity estimate: 0bit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh           0bit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

qdisc cake 40: parent 1:40 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 0b of 4Mb
 capacity estimate: 0bit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh           0bit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

qdisc cake 20: parent 1:20 bandwidth unlimited besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0 mpu 84 no-sce
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 0b of 4Mb
 capacity estimate: 0bit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh           0bit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

Thank you all for getting into this, I truly appreciate the insights.

chromi commented 3 years ago

On 9 Jul, 2021, at 11:54 pm, Bruno Bergami @.***> wrote:

Using startup script as always. Usually worked. Not this time.

Okay, disable both sets of startup scripts, and try running your script directly from the terminal instead. That will let you see any error messages. Stick with the version I suggested, for now.

xnoreq commented 3 years ago

@brbergami You need to add those 2> /dev/null at the end of the same line as each command. It redirects error output of the command. But remove this redirection altogether to see the errors...

Don't you have access to the router via SSH? Then you could run these commands interactively and see where it goes wrong. "It didn't work" is not extremely useful. But yes, you should see cake instead of mq+fq_codels.

brbergami commented 3 years ago

Tried to set this manually line per line.

First 3 del commands run without problem.

Set egress seems to be ok, since no errors were given.

But when tried to do ip link add name ifb4eth0 type ifb, I get the error:

ip: RTNETLINK answers: Not supported So I have to do modprobe ifb before that.

Resetted and I did everything again just to be sure...

No errors given except when entering these lines:

tc qdisc del dev eth0 ingress
tc qdisc del dev ifb4eth0 root

For those I'm getting RTNETLINK answers: No such file or directory. Continued, and this is the status:

tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8001: dev eth0 root refcnt 9 bandwidth 24000Kbit diffserv4 dual-srchost nat nowash ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64 no-sce
 Sent 2460745 bytes 22875 pkt (dropped 32, overlimits 10021 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 43520b of 4Mb
 capacity estimate: 24000Kbit
 min/max network layer size:           28 /    1500
 min/max overhead-adjusted size:       64 /    1518
 average network hdr offset:           14

                   Bulk  Best Effort        Video        Voice
  thresh       1500Kbit    24000Kbit    12000Kbit     6000Kbit
  target         12.1ms        5.0ms        5.0ms        5.0ms
  interval      107.1ms      100.0ms      100.0ms      100.0ms
  pk_delay          0us        748us          0us         82us
  av_delay          0us         36us          0us          4us
  sp_delay          0us          2us          0us          2us
  backlog            0b           0b           0b           0b
  pkts                0        22872            0           35
  bytes               0      2458612            0         4245
  way_inds            0           12            0            0
  way_miss            0          130            0            2
  way_cols            0            0            0            0
  marks               0            0            0            0
  drops               0            0            0            0
  ack_drop            0           32            0            0
  sp_flows            0            1            0            0
  bk_flows            0            1            0            0
  un_flows            0            0            0            0
  max_len             0         7782            0          157
  quantum           300          732          366          300

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth1 root
 Sent 2811733 bytes 5106 pkt (dropped 0, overlimits 0 requeues 36)
 backlog 0b 0p requeues 36
qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 2811733 bytes 5106 pkt (dropped 0, overlimits 0 requeues 36)
 backlog 0b 0p requeues 36
  maxpacket 7570 drop_overlimit 0 new_flow_count 66 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev wlan0 root
 Sent 117924761 bytes 88777 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev wlan0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 117923368 bytes 88770 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 3028 drop_overlimit 0 new_flow_count 280 ecn_mark 0
  new_flows_len 1 old_flows_len 3
qdisc fq_codel 0: dev wlan0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 1393 bytes 7 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8002: dev ifb4eth0 root refcnt 2 bandwidth 264000Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64 no-sce
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 0b of 13200000b
 capacity estimate: 264000Kbit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh     264000Kbit
  target          5.0ms
  interval      100.0ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  marks               0
  drops               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

Seems like the ifb is not working. Did navigation just to generate some stats, but no luck.

chromi commented 3 years ago

On 12 Jul, 2021, at 12:58 am, Bruno Bergami @.***> wrote:

Seems like the ifb disk is not working. Did navigation just to generate some stats, but no luck.

Okay, looks like you have some improvement. It also looks like traffic is not reaching the ingress portion of the Ethernet device, and therefore not being redirected to the IFB device. This might be due to the forwarding accelerator still being active, so try disabling that.

brbergami commented 3 years ago

It's not enabled, if you're talking about SFE.

The IFB shows status UNKNOWN, but the last line tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb4eth0 should enable it or leave me disconnected from ingoing traffic, but I'm still connected. No clue there.

xnoreq commented 3 years ago

Well obviously something is wrong with your eth0 ingress. Try: tc qdisc del dev eth0 handle ffff: ingress

brbergami commented 3 years ago

Btw, modprobe act_mirred fails with:

modprobe: module act_mirred not found modprobe: failed to load module act_mirred: No such file or directory

Could be that the culprit for the last command (tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb4eth0) to not work properly?

brbergami commented 3 years ago

tc qdisc del dev eth0 handle ffff: ingress

Just tried that. Getting RTNETLINK answers: No such file or directory

chromi commented 3 years ago

On 12 Jul, 2021, at 3:49 am, Bruno Bergami @.***> wrote:

Btw, modprobe act_mirred fails with:

modprobe: module act_mirred not found modprobe: failed to load module act_mirred: No such file or directory

Could be that the culprit for the last command (tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb4eth0) to not work properly?

Well, it certainly won't help. That is a problem you'll need to solve to get the system working correctly, even if it's not the only problem.

brbergami commented 3 years ago

My question there is if module act_mirred is required for cake to mirror to the wan interface 🙃

On 12 Jul, 2021, at 3:49 am, Bruno Bergami @.**> wrote: Btw, modprobe act_mirred fails with: modprobe: module act_mirred not found modprobe: failed to load module act_mirred: No such file or directory Could be that the culprit for the last command (tc filter add dev eth0 parent ffff: matchall action mirred egress redirect dev ifb4eth0) to not work properly? Well, it certainly won't help. That is a problem you'll need to solve to get the system working correctly, even if it's not the only* problem. - Jonathan Morton

chromi commented 3 years ago

On 12 Jul, 2021, at 4:11 am, Bruno Bergami @.***> wrote:

My question there is if module act_mirred is required for cake to mirror to the wan interface 🙃

It is required for ingress shaping. That's what the error message is telling you.

brbergami commented 3 years ago

Nevermind. I figured out. Everything is working now.

brbergami commented 3 years ago

Msg for you, the folk from the future who's crawling the internet to get this answer:

If you're trying to set custom Cake in DD-WRT, this is the startup script you need:

sleep 60;

#If required
modprobe ifb
modprobe sch_cake

#Egress
tc qdisc add dev eth0 root cake egress ethernet diffserv4 dual-srchost bandwidth 24mbit

#Ingress
ip link add name ifb4eth0 type ifb
tc qdisc del dev eth0 ingress
tc qdisc add dev eth0 handle ffff: ingress
tc qdisc del dev ifb4eth0 root
tc qdisc add dev ifb4eth0 root cake ethernet diffserv4 dual-dsthost wash bandwidth 315mbit
ip link set ifb4eth0 up
tc filter add dev eth0 parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth0

Obviously, adjust the bandwidth and other settings to your need. The problem for me was what was mirrored to the ifb device.

kernel-panic69 commented 3 years ago

On 12 Jul, 2021, at 4:11 am, Bruno Bergami @.***> wrote: My question there is if module act_mirred is required for cake to mirror to the wan interface 🙃 It is required for ingress shaping. That's what the error message is telling you. - Jonathan Morton

CONFIG_NET_ACT_MIRRED is compiled into the kernel, not as a module:

https://svn.dd-wrt.com/browser/src/linux/universal/linux-4.9/.config_mvebu#L1111

@brbergami thanks for this.

dtaht commented 2 years ago

I always appreciate statistics and tests as to how well cake is working, btw.