dtaht / sch_cake

Out of tree build for the new cake qdisc
101 stars 35 forks source link

Make tin bandwidth threshold configurable #130

Open ckkcc opened 4 years ago

ckkcc commented 4 years ago

It'd be useful to be able to configure the thresholds, e.g. in some scenarios, I want to allow higher priority traffic to completely starve lower ones (e.g. allow video call traffic to starve browser traffic), so I'd set 100% threshold to all tins to enforce strict priority which would work like a prio qdisc

It'd also be useful to allow flexible arrangement of tins vs flows, e.g. to discourage Diffserv misuse, the traffic could be divided per host first, then into tins for each host, then into flows for each tin.

dtaht commented 4 years ago

the hard coding of the levels here is an outgrowth of cakes design. John Nagle once said, philosophically, "that every application had a right to one packet in the network", and I really believe in that. I think strict priority queues are easily misapplied and are dangerous to the health of the network. Let me give you two examples - arp is something you only rarely think about, but when it fails, so does the network. It's easy with a strict priority queue to forget about that - or ipv6 RA. I've done it! Similarly, I use a routing protocol between links. Mine happens to use CS6, which helps, but it doesn't use ECN, which means you can starve it too, which is a subject of a long philosophical debate I have no time for here.

Being fair seems to be a win across the board. Being slightly unfair as the diffserv support allows you to be while always allowing some bandwidth for all classes, seems to help, also.

However: Cake works from "models" of which there are 4 presently - besteffort, 3 tier, 4 tier, and precidence (the last should not be used) out of which fairness is applied essentially first. If you would like to write a different model with different thresholds and present a use case for it, it is straightforward. https://github.com/dtaht/sch_cake/blob/master/sch_cake.c#L2539

As for prioritizing individual hosts. Well, some folk would really really like that. Same folk insist on selling their features so that say netflix is prioritized, or gaming prioritized, and per host per flow fq really really just generally kills that problem, invisibly. People that want it, I tend to ask to just go turn on cake and measure, first.

That said, it is feasible to do that in htb or in adding yet another layer of "drr" to the existing algorithms on a per host basis so you could do weighted fq over that.

ckkcc commented 4 years ago

Before I came across Cake, I used various qdisc including drr, tbf, prio, hfsc/htb to shape the traffic. When I applied strict priority to the traffic, I only put low bandwidth latency-sensitive traffic in strict high priority queues which most of the time wouldn't starve lower priority traffic, starvation only occurred during temporary spikes which was desirable. While mis-configured strict priority queues could be detrimental to the health of the network, the feature nevertheless is useful when it's done properly.

Besides, the default qdisc pfifo_fast also uses strict priority queues, and Cake Best Effort tin can also starve Bulk tin, one could already mis-configure the network by giving file sharing traffic Best Effort priority or higher and everything else Bulk priority.

One OpenWrt user has a similar use case here: https://forum.openwrt.org/t/sqm-3-layer-cake-with-br-and-ip-tables-packet-marking/12256/7

dtaht commented 4 years ago

bulk tin gets 5%.

I never much cared for pfifo_fasts behavior, either. Easy to disable if you tried. surprising so few have tried.

submit a model if you wish.

dtaht commented 4 years ago

I spent a bit of time thinking about this. There are two strategies than can be pursued that are slightly more amiable to "better voip". One is a hard deadline or pure drop head scheduler. If a packet of a voip flow cannot be delivered within the sample rate (say 2.7ms), drop it and deliver the next one, don't be as gentle as codel.

also you should take a look at the cake four tier A/V model. With strict admission control you can get a lot more room for voip. A two tier model could be derived from that, but to fully impement the behavior I think you want, I would still change the scheduler to be packet, not quantum, oriented, nor apply codel to the voip queue... and at that point it's no longer cake.

I tend to care cake works well with videoconferencing, which it does, and the existing behaviors are tcp-friendly, where, for example, the hard head drop I suggested couldn't possibly work for video.

yutayu commented 4 years ago

I have an opinion. Cake should be planned by only efficiency for packet habit like fq_codel, if you want to divide bandwidth each class, you should use htb+ leaf cake qdisc.

micron10 commented 4 years ago

but how? do you have example

On Mon, Aug 10, 2020, 16:15 yutayu notifications@github.com wrote:

I have an opinion. Cake should fit efficiency for packet habit, if you want to divide bandwidth each class, you should use htb+ leaf cake qdisc.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dtaht/sch_cake/issues/130#issuecomment-671346978, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCWD5HQ4ECQKNN2O6PN6UTR77XHHANCNFSM4MGBMPOA .

yutayu commented 4 years ago

@micron10 https://wiki.archlinux.org/index.php/Advanced_traffic_control I use a readymade sample.

tc qdisc add dev eth0 root handle 1: htb default 30 tc class add dev eth0 parent 1: classid 1:1 htb rate 6mbit burst 15k tc class add dev eth0 parent 1:1 classid 1:10 htb rate 5mbit burst 15k tc class add dev eth0 parent 1:1 classid 1:20 htb rate 3mbit ceil 6mbit burst 15k tc class add dev eth0 parent 1:1 classid 1:30 htb rate 1kbit ceil 6mbit burst 15k tc qdisc add dev eth0 parent 1:10 handle 10: cake besteffort tc qdisc add dev eth0 parent 1:20 handle 20: cake besteffort tc qdisc add dev eth0 parent 1:30 handle 30: cake besteffort

I modified last 3 lines. You can add any filters(destinations,ports,and so on) to classfy packets and priorize each class. Cake adapts each class insted of sfq in this case. This is what I wanted to say.

Edit, Cake can add filter.

dim-geo commented 3 years ago

I have a proposal that could be used as a middle ground. Right now, cake in diffserv8 uses (tin_number * 7/8) rate of each tin. You could add a simple configurable integer option in cake to control the slope of the rates.

For example: change the calculation to: tin_number * slope/32. Default value of slope would be 28. Maximum value could be 31. If user wants, he can set factor as 31/32 (96%), which is very flat and would make cake behave almost like htb but prevent total starving of lower queues.

dtaht commented 2 years ago

It's been a while since I thought about this. I was not aware of the use case an ISP might have in the middle of their network. Does anyone else care?

dim-geo commented 2 years ago

Personally, I migrated away from cake (diffserv8 + fw marks) to hfsc+fq_codel due to this. I have a limited upload (1Mbit) which is saturated by torrent upload. Remote desktop was suffering, because cake allowed enough bandwidth to be given on bulk. large-size packets are not allowed. classes: guest wifi,torrent,bulk web,web,video+remote desktop,voip+dns

Other option was to move remote desktop priority lower to bulk web, but I didn't like that, because a bulk transfer could mess up my remote desktop.

moeller0 commented 2 years ago

I would respectfully propose you had way too many priority tiers for a 1 Mbps link... Just go with three from low to high: 1) bulk: low priority scavenger/background for torrent traffic only 2) best effort: web browsing and everything else 3) remote desktop

(so either diffserv3 or diffserv4 with the highest class not used). The challenge is to get a higher priority class that has enough guaranteed throughput to allow your remote desktop use case... IIRC in diffserv4 the "Video" class gets 50% of the link capacity which would be 0.5 Mbps, would that be enough?

The trick about prioritization IMHO is to use it sparingly. For every packets that gets serviced with lower latency, some other packets will need to be delayed, if there are too few packets that can be delayed your higher priority traffic is going to delay itself.

On May 31, 2022, at 11:57, dim-geo @.***> wrote:

Personally, I migrated away from cake (diffserv8 + fw marks) to hfsc+fq_codel due to this. I have a limited upload (1Mbit) which is saturated by torrent upload. Remote desktop was suffering, because cake allowed enough bandwidth to be given on bulk. large-size packets are not allowed. classes: guest wifi,torrent,bulk web,web,video+remote desktop,voip+dns

Other option was to move remote desktop priority lower to bulk web, but I didn't like that, because a bulk transfer could mess up my remote desktop.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.