How to set download/upload limit per IP/MAC of device?

[gazambuja]

Im using sqm-scripts over Openwrt, the general limits on my guest interface work fine, but how I can set another limit per host and not only general limits?

Something like:

config queue option enabled '1' option interface 'br-guest' option download '3000' option upload '10000' ....

config devices option enabled '1' option mac-addr 'xx:xx:xx:xx:xx' option download '800' option upload '1500' .... config devices option enabled '1' option mac-addr 'all' option download '500' option upload '1000' ....

Thanks

[tohojo]

sqm-scripts has no support for this kind of per-host limiting. Are you using cake as the shaper? If so you can get per-host fairness by adding the right parameters in the newest version...

[EricLuehrsen]

In concept its possible with the current standard kernel shapers and qdisc, but it would be a lot of scripting work. Lets assume we only want to control access to the WAN for a host, and leave local communication alone. Generically (cake or no), there could be a way to assign a subclass shaper with its own qdisc and filter on IP (tc filter ~ u32 match src / dst) or MAC through contrack. A generic functions.sh subroutine would need to have the shaper as a parameter and know how to handle each shaper. The qdisc is already a dynamic option. As long as all scripts followed class numbering convention, this could be a plugin for all of them from functions.sh. It gets ugly fast though. CPU load for each class ingress and egress. The code to unfurl UCI is not a small effort. IPV4 address white/black list from DHCP is easy. But IPV6 has so many permutations. So you can only have a single router so that contrack can hold the mac addresses. Multi-router subnets and you don't know who is who with SLAAC privacy.

[tohojo]

Yes, and the added complexity is definitely out of scope for sqm-scripts. Closing this issue; I do believe the qos-scripts package can do (some of) what you are describing...

[chros73]

@tohojo : "Are you using cake as the shaper? If so you can get per-host fairness by adding the right parameters in the newest version..." What feature do you refer to? I couldn't find any doc about this. (I just compiled OpenWRT 15.05.1 with kmod_sched_cake) Thanks

[tohojo]

chros73 notifications@github.com writes:

@tohojo : "Are you using cake as the shaper? If so you can get per-host fairness by adding the right parameters in the newest version..." What feature do you refer to? I couldn't find any doc about this. (I just compiled OpenWRT 15.05.1 with kmod_sched_cake)

Well, the documentation is somewhat lacking, I guess. Basically, using the parameters introduced with this commit: https://github.com/dtaht/tc-adv/commit/4adbb88809bfa66422de29beb8b70f2e4b56250a (see the comments too) you can get cake to treat hosts separately. There's no explicit support for this in the SQM GUI, though...

-Toke

[moeller0]

On Jun 3, 2016, at 10:41 , chros73 notifications@github.com wrote:

@tohojo : "Are you using cake as the shaper? If so you can get per-host fairness by adding the right parameters in the newest version..." What feature do you refer to? I couldn't find any doc about this. (I just compiled OpenWRT 15.05.1 with kmod_sched_cake) Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

Try: tc qdisc add cake help on your router. Here is the output on mine: Usage: ... cake [ bandwidth RATE | unlimited* | autorate_ingress ] [ rtt TIME | datacentre | lan | metro | regional | internet* | oceanic | satellite | interplanetary ] [ besteffort | precedence | diffserv8 | diffserv4* ] [ flowblind | srchost | dsthost | hosts | flows* | dual-srchost | dual-dsthost | triple-isolate ] [ atm | noatm* ] [ overhead N | conservative | raw* ] [ wash | nowash* ] [ memlimit LIMIT ](* marks defaults)

The relevant line is: [ flowblind | srchost | dsthost | hosts | flows* | dual-srchost | dual-dsthost | triple-isolate ]

dual-srchost, dual-dsthost and triple-isolate promise to implement a hierarchical fairness first over IPs and then foe each IP over flows. The caveat currently is that cake needs to see the real addresses, but due to the typical NAT performed on WAN interfaces the relevant information about the internal IPs is hidden, or better due to NAT cake only sees one internal IP address and the by IP-fairness does not result in what one woukd expect. One work around is to instantiate cake on the internal interface of the router (typically connected to a switch) there cake sees the real internal IP addresses. If you should attempt to try this, please note that the meaning of download and upload for SQM-scripts is always from the point of view of the router and not the internet, that means on an internal interface the routers ingress/download direction is the end hosts egress/upload direction, so adjust the bandwidths accordingly. In that situation I believe on ingress you should set the: “Advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully” field to dual-dsthost and the “Advanced option string to pass to the egress queueing disciplines; no error checking, use very carefully” to dual-srchost. In recent testing such a setup worked okay, while triple-isoloate on both ingress and egress did not work (even though it should have). Make sure to also select either layer_cake or piece_of_cake as script and cake as qdisc. Please note that you need to clear the advanced option strings if you use fq_codel, otherwise they will fail. Once the isolation options actually proofed useful and robust, I plan to give them a better treatment in the GUI, but they are still under-tested/WIP.

Best Regards

[moeller0]

Hi Toke,

On Jun 3, 2016, at 10:49 , Toke Høiland-Jørgensen notifications@github.com wrote:

chros73 notifications@github.com writes:

@tohojo : "Are you using cake as the shaper? If so you can get per-host fairness by adding the right parameters in the newest version..." What feature do you refer to? I couldn't find any doc about this. (I just compiled OpenWRT 15.05.1 with kmod_sched_cake)

Well, the documentation is somewhat lacking, I guess. Basically, using the parameters introduced with this commit: https://github.com/dtaht/tc-adv/commit/4adbb88809bfa66422de29beb8b70f2e4b56250a (see the comments too) you can get cake to treat hosts separately. There’s no explicit support for this in the SQM GUI, though...

I believe after testing that it works as intended we should expose that in the GUI. Currently it looks like it does not work with NAT, which is unfortunate because that makes is basically un-attractive for users that can not dedicate a router for shaping purposes*…

-Toke — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[chros73]

Thanks for the quick response from all of you, and yes, it's not attractive at all in this way :) (I have read your, @moeller0 , full explanation in this thread: https://forum.openwrt.org/viewtopic.php?id=64812&p=2)

I still prefer the idea in the OP with realization of @EricLuehrsen's idea. (I came across with this patch for simple.qos https://gist.github.com/1player/fba5775fe0780af310a3 , and it uses the same idea but only for upload.)

Can you help me quickly modify e.g. the existing layer_cake.qos (since it's the easiest) (by creating a new one e.g. layer_cake_limit.qos on a branch/fork/etc.) with the following prerequisities (using @EricLuehrsen's idea in this degraded form):

• we only have a single router
• we only care about IPv4 addresses (for now)
• we only want to control access to the WAN for only a single subnet (let's say 192.168.10.0/24) (for now)
• we want to limit bandwidth of the given upload/download rates for these hosts to: uplink ${UPLINK}/2 , downlink ${DOWNLINK}/2
• create a new qdisc for these
• and filter on IP (tc filter ~ u32/24/etc. match src / dst)
• everything can be hardcoded in this new script for now

I also try to create this, but I have really limited networking knowledge. :) Thanks for thinking with me!

[tohojo]

While I don't want to discourage experimentation I note that anything requiring per-host configuration of filters or the like is probably out of scope for inclusion in sqm-scripts. We have deliberately tried to make it a per-interface thing with only the most basic knobs. Big complicated QoS settings etc is what qos-scripts is for... ;)

[chros73]

anything requiring per-host configuration of filters or the like is probably out of scope for inclusion in sqm-scripts.

I understand it completely. That's why I haven't reopened this issue :)

We have deliberately tried to make it a per-interface thing with only the most basic knobs.

I understand this and appreciate it! :)

[moeller0]

Hi chros73,

On Jun 3, 2016, at 14:18 , chros73 notifications@github.com wrote:

Thanks for the quick response from all of you, and yes, it's not attractive at all in this way :) (I have read your, @moeller0 , full explanation in this thread: https://forum.openwrt.org/viewtopic.php?id=64812&p=2)

I still prefer the idea in the OP with realization of @EricLuehrsen's idea. (I came across with this patch for simple.qos https://gist.github.com/1player/fba5775fe0780af310a3 , and it uses the same idea but only for upload.)

Can you help me quickly modify e.g. the existing layer_cake.qos (since it's the easiest) (by creating a new one e.g. layer_cake_limit.qos on a branch/fork/etc.) with the following prerequisities (using @EricLuehrsen's idea in this degraded from):

• we only have a single router • we only care about IPv4 addresses (for now) • we only want to control access to the WAN for only a single subnet (let's say 192.168.10.0/24) (for now) • we want to limit bandwidth of the given upload/download rates for these hosts to: uplink ${UPLINK}/2 , downlink ${DOWNLINK}/2 • create a new qdisc for these • and filter on IP (tc filter ~ u32/24/etc. match src / dst) • everything can be hardcoded in this new script for now I also try to create this, but I have really limited networking knowledge. :) Thanks for thinking with me!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

I am with Toke on this. I have one thing I want to test, that might actually help in your case a bit, would be a white-list of IPs used to delegate al traffic to and from into the background priority tier. The idea is to use dscp BK1 marks to allow to de-prioretize traffic for specific IPs/subnets simply to allow a work-around for the problem flow-fair queueing has with flow-rich applications like bit torrent. In case cakes bit torrent control starts to work reliably and robustly, I might not even want to implement the proposed hack.

Best Regards

[moeller0]

On Jun 3, 2016, at 14:22 , Toke Høiland-Jørgensen notifications@github.com wrote:

While I don't want to discourage experimentation I note that anything requiring per-host configuration of filters or the like is probably out of scope for inclusion in sqm-scripts. We have deliberately tried to make it a per-interface thing with only the most basic knobs. Big complicated QoS settings etc is what qos-scripts is for... ;) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Hi Toke,

I agree, but how open would you be for a collection of IPs whose packets to re-mark as CS1/BK? Since most bit torrent clients do not allow to set DSCP marks users have ne real chance of putting these nicely into the BK tier of simple and layer_cake, so re-marking them CS1 before handing them to sqm would allow some crude work arounds. This will be easy for egress, but for ingress it has the same NAT issue as we have already. And it will be a challenge to make this work with IPv6 privacy addresses…

Best Regards

[tohojo]

moeller0 notifications@github.com writes:

I agree, but how open would you be for a collection of IPs whose packets to re-mark as CS1/BK?

Depends on the complexity of the implementation, both in terms of UI and changes to the scripts. :)

-Toke

[moeller0]

On Jun 3, 2016, at 15:10 , Toke Høiland-Jørgensen notifications@github.com wrote:

moeller0 notifications@github.com writes:

I agree, but how open would you be for a collection of IPs whose packets to re-mark as CS1/BK?

Depends on the complexity of the implementation, both in terms of UI and changes to the scripts. :)

So no conceptual stumbling blocks, just an implementation complexity issue then. Let me see what I come up with and if to complex let’s burry the idea again. My idea is to use have a simple entry field to put a say space separated list of to be deprioritized IPs, use ipset to turn these into a single iptables entity and then use this entity and iptables to remark packets. Which will only work for egress, due to NAT and the fact that on ingress the ifb runs before netfilter code… But let’s see whether that works at all…

Best Regards

-Toke — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[tohojo]

Well, it is a slippery slope towards more configuration complexity, but if it can be kept simple enough it may be worth adding.

As far as the implementation is concerned, I'm particularly worried about things that need to be supported in all scripts (because then we need to change them all and keep them up to date with all such new features). Not sure if we can come up with a mechanism to avoid that, though. So let's see how it turns out and evaluate...

[dtaht]

A note: qos-scripts was completely broken for ipv6 last I looked, which was, admittedly, years ago.

[dtaht]

Another note: I really wish any benefit from the hfsc versions of these had been carefully quantified. Looking at the added complexity now, what should a new user chose?

[moeller0]

Hi Toke, The first Let's See how this turns out....

On June 3, 2016 4:33:31 PM GMT+02:00, "Toke Høiland-Jørgensen" notifications@github.com wrote:

Well, it is a slippery slope towards more configuration complexity, but if it can be kept simple enough it may be worth adding.

Currently I envision a simple list of internal IP to demote all packets of to CS1.

As far as the implementation is concerned, I'm particularly worried about things that need to be supported in all scripts (because then we need to change them all and keep them up to date with all such new features). Not sure if we can come up with a mechanism to avoid that, though. So let's see how it turns out and evaluate...

Well only Shaper's offering DSCP based tiering will be specials, and we could try to hook this into the generic start script somehow. But I get ahead of myself, let me come up with a prototype and test it first...l

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/tohojo/sqm-scripts/issues/23#issuecomment-223595442

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[dtaht]

Moving on to the "per host, post nat" issue:

A) the need for extensive classification is partially a perception, and partially a human need for control, and partially a real need.

I am always pushing for people to actually measure their performance after applying the most basic of fq_codel (please note I'm not saying cake here because I fear the aqm portion is actually busted again due to lack of testing) - and then see what is not working as well as they would like - and share the measurement. As one example, under fq_codel derived algorithms, web traffic cuts through uTP torrent traffic like butter in the general case. Gaming traffic only sees issues on large torrents on the download side where we have more limited control over what we see.

B) with nat, we are doing per flow (essentially by port number as the internal ip is not available) fq and aqm in the general case, which essentially is no different than the non-nat behavior. But: folding these into a tier of QoS is made more difficult because there is not an identifiable port number to use, and dscp markings are not guaranteed to be preserved e2e.

C) on outbound we have extensive control, and can, for example, do all sorts of tricks on our internal applications (like configure a local torrent client for a bandwidth limit and (at least transmission supports this) setting a default dscp marking point that we can easily take advantage of. Same goes for things such as dns, rsync, and so on.

D) on inbound, far less so. (still, in the case of torrents, IF you have control of your clients, you can configure them to be less annoying, use a schedule, etc). A few ISPs expose a QoS interface on their end to give clients a better shot at controlling their traffic.

E) IMQ is better than ifb for some purposes. The flow relationship between ifb, tc and iptables has always been problematic as iptables is too late in the stack. Adding any degree of iptables functionality directly into cake seems painful.

F) I have been watching the "bpf on rx" work with some eagerness: https://lwn.net/Articles/682290/ as this may offer a partial solution to the reclassification problem we have, and also hoping that one day we could hook cake into the inbound path without tc_mirred.

[chros73]

@moeller0 : interesting idea, thanks, it can worth a shot.

Let me summarize the 3 options that we talked about in this issue and their pros and cons. (Please correct me if I'm wrong.)

I. @moeller0's idea: deprioritized IPs

• cons: only (?) works for egress;
• If I understand it correctly: will All the packets of the given IPs be deprioritized? That would mean there won't be any highly prioritized flow (like http, etc...)
• pros: no fix bandwidth limit (!), we won't need any extra logic in the scripts

II. @EricLuehrsen's idea: using additional qdiscs and filters

• cons: fix bandwidth limit (!) ; extra logic in the scripts; decreased performance (? thanks to the additional qdiscs)
• And that make me think how the 2 cake qdiscs would behave (as was in my proposal) ? @dtaht Is it something even worth to bother with?
• pros: it's working for both ingress and egress

III. @gazambuja's idea: put them into an interface then limit that whole interface

• cons: fix bandwidth limit (!) ; decreased performance (? thanks to the additional qdiscs); the bandwidth of the whole interface is limited
• pros: it's working for both ingress and egress; we won't need any extra logic in the scripts (so it won't raise the question in the II.)

[moeller0]

Hi Dave,

On June 3, 2016 4:45:24 PM GMT+02:00, "Dave Täht" notifications@github.com wrote:

A note: qos-scripts was completely broken for ipv6 last I looked, which was, admittedly, years ago.

This was addressed recently and hopefully is fixed now. I have not tested it myself as my IPv6 went missing....

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/tohojo/sqm-scripts/issues/23#issuecomment-223598744

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[moeller0]

Hi Dave,

On June 3, 2016 4:46:49 PM GMT+02:00, "Dave Täht" notifications@github.com wrote:

Another note: I really wish any benefit from the hfsc versions of these had been carefully quantified. Looking at the added complexity now, what should a new user chose?

    Whatever they please? The big advantage of our script framework is that one can easily add new script. We could improve our default and documentation a bit, there was some activity around that, but it subsided before reaching real results....

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/tohojo/sqm-scripts/issues/23#issuecomment-223599141

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[moeller0]

Hi Dave,

On Jun 3, 2016, at 16:45 , Dave Täht notifications@github.com wrote:

A note: qos-scripts was completely broken for ipv6 last I looked, which was, admittedly, years ago.

There were commits recently that aimed at fixing this deficiency, I have not tested those, but have no reason t believe that qos-scripts still has an IPv6 blind-spot.

Best Regards

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[moeller0]

Hi Dave,

On Jun 3, 2016, at 17:18 , Dave Täht notifications@github.com wrote:

Moving on to the "per host, post nat" issue:

A) the need for extensive classification is partially a perception, and partially a human need for control, and partially a real need.

I am always pushing for people to actually measure their performance after applying the most basic of fq_codel (please note I'm not saying cake here because I fear the aqm portion is actually busted again due to lack of testing) - and then see what is not working as well as they would like - and share the measurement. As one example, under fq_codel derived algorithms, web traffic cuts through uTP torrent traffic like butter in the general case. Gaming traffic only sees issues on large torrents on the download side where we have more limited control over what we see.

So I am trying to shepherd people in the openwrt forum who encounter sqm-script issues. I have a number of people reporting that they have control over the router, but also have a number of users on their network that can not seem to be bothered to teach their BT clients to behave (or have clients that lack proper configurability. I typically try to get them to test the standard scripts we recommend, often ameliorating the issue. But there is a number of cases where well meaning BT users use UDP-based µTP and other people use UDP-based online games, and that combination does not seem to work well. Especially since sqm basically removes the latency increase under load exactly the signal µTP relies on to throttle itself back. I hope that offering a single additional manual intervention method to quarantine the complete traffic to/from specific internal hosts into the low priority tier might help here. Not perfect, but at least a simple centralized way to deal with unruly/uncooperative co-inhabitants… As it is, bit torrent os the major traffic type that htb/fq_codel scripts have problems controlling well.

B) with nat, we are doing per flow (essentially by port number as the internal ip is not available) fq and aqm in the general case, which essentially is no different than the non-nat behavior.

Put differently at that point we have no idea about the internal IPs and hence can not easily enforce per IP-fairness.

But: folding these into a tier of QoS is made more difficult because there is not an identifiable port number to use, and dscp markings are not guaranteed to be preserved e2e.

As you encountered with comcast ingress DSCP marks are not to be trusted generally.

C) on outbound we have extensive control, and can, for example, do all sorts of tricks on our internal applications (like configure a local torrent client for a bandwidth limit and (at least transmission supports this)

That seems to be an issue for some router administrator in that they need to work with non-cooperative internal host operators), so some level of configuration on the router is occasionally requested.

setting a default dscp marking point that we can easily take advantage of. Same goes for things such as dns, rsync, and so on.

In theory internal host should be able to at least mark their outgoing packets with appropriate DSCP markings; BUT windows requires a domain controller for that to work (but does allow DSCP assignment on a per-application basis if need be) and Linux can do this with iptables, but to my knowledge no distribution offers easy (GUI) access to such a setting. Also that leaves inbound unsolved, arguably the bigger ‘half” of the problem space…

D) on inbound, far less so. (still, in the case of torrents, IF you have control of your clients, you can configure them to be less annoying, use a schedule, etc). A few ISPs expose a QoS interface on their end to give clients a better shot at controlling their traffic.

Agreed, so there definitely is a need to get a handle on ingress, but I shudder at te thought of using tc to implement a dscp-by-internal-IP re-mapper, even though such a beast would be really helpful.

E) IMQ is better than ifb for some purposes. The flow relationship between ifb, tc and iptables has always been problematic as iptables is too late in the stack. Adding any degree of iptables functionality directly into cake seems painful.

I note, there has been rumblings on the netdev mailing list of getting netfilter/iptables hooks into the ingress qdisc, promising that one day we might be able to use ipsets and friends on ingress as well. As far as I can tell the only thing cake/htb/fq_codel would need to be able is see the internal IP addresses/ports optionally instead of just the NATed ones, not more, but also not less.

F) I have been watching the “bpf on rx" work with some eagerness: https://lwn.net/Articles/682290/ as this may offer a partial solution to the reclassification problem we have, and also hoping that one day we could hook cake into the inbound path without tc_mirred.

If I remember correctly getting rid of the ingress qdisc (by setting up sqm on the egress side of a routers externally- and internally facing interfaces) only recovered around 5% more bandwidth under saturating load (with a true wan bandwidth >> the routers shaping capability). So getting rid of the tc_mirred is not bad, but also not a big help performance wise.

Best Regards

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[moeller0]

Hi Dave,

On Jun 3, 2016, at 16:46 , Dave Täht notifications@github.com wrote:

Another note: I really wish any benefit from the hfsc versions of these had been carefully quantified. Looking at the added complexity now, what should a new user chose?

So the upshot of the discussion and measurements at the time was, that HTB and HFSC/cake have different CPU overload behavior, HTB will keep the latency under load close to the specified target, but will sacrifice more bandwidth, while the other two will compromise latency but keep the bandwidth closer to the specified shaping rates. Exercising WLAN/WIFI traffic is a great way to drive puny home routers into CPU overload. Personally I prefer to keep the latency sane, but a lot of users seem to prefer a compromise with less bandwidth sacrifice.

That said a new user should obviously just stick to the default script: simple.qos. Our help texts might require tweaking. But overall choice is good ;)

Best Regards

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chros73]

@dtaht : Looking at the added complexity now, what should a new user chose?

There's a test on OpenWRT WIKI (including hfsc_lite , hfsc_litest): https://wiki.openwrt.org/doc/howto/sqm#selecting_the_optimal_queue_setup_script

[chros73]

Can you help me quickly modify e.g. the existing layer_cake.qos (since it's the easiest) (by creating a new one e.g. layer_cake_limit.qos on a branch/fork/etc.) with the following prerequisities (using @EricLuehrsen's idea in this degraded form):

• we only have a single router
• we only care about IPv4 addresses (for now)
• we only want to control access to the WAN for only a single subnet (let's say 192.168.10.0/24) (for now)
• we want to limit bandwidth of the given upload/download rates for these hosts to: uplink ${UPLINK}/2 , downlink ${DOWNLINK}/2
• create a new qdisc for these
• and filter on IP (tc filter ~ u32/24/etc. match src / dst)
• everything can be hardcoded in this new script for now

I have created something (I haven't tried it though until I'm sure it's good): https://github.com/chros73/sqm-scripts/commit/77d54370bd79d4143e02d551516bda7834aecbd3 Can you take a look at it and please bear with me :)

[moeller0]

Chros73,

On Jun 3, 2016, at 20:40 , chros73 notifications@github.com wrote:

Can you help me quickly modify e.g. the existing layer_cake.qos (since it's the easiest) (by creating a new one e.g. layer_cake_limit.qos on a branch/fork/etc.) with the following prerequisities (using @EricLuehrsen's idea in this degraded form):

• we only have a single router • we only care about IPv4 addresses (for now) • we only want to control access to the WAN for only a single subnet (let's say 192.168.10.0/24) (for now) • we want to limit bandwidth of the given upload/download rates for these hosts to: uplink ${UPLINK}/2 , downlink ${DOWNLINK}/2 • create a new qdisc for these • and filter on IP (tc filter ~ u32/24/etc. match src / dst) • everything can be hardcoded in this new script for now I have created something (I haven’t tried it though until I'm sure it's good): chros73@77d5437

Why, you have the code, you have a real world test case I presume, so do not be shy test whether you modification works or not. Please report success or failure back. 

Can you take a look at it and please bear with me :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[EricLuehrsen]

Chros73, Thanks but that really wasn't my idea. Gargoyle and many routers factory firmware will more or less do this already (though with derivatives of earlier QOS flavors). I was merely expressing the burdens of qdisc experimentation and quota-control in one project. As far as your specific use case, the pattern in simple.qos or hfsc_lite.qos maybe easier to follow. Just copy one to my_subnet.qos and add another class and filter. Don't forget to read the documentation for "tc show..." so you can view a few things on board. Eric

[braian87b]

Hi, can sqm-scripts be used side-by-side simnultaneously with qos-script package to accomplish this? sqm-scripts to protect from bufferbloat and qos-script to throttle the users speed or even the websites speed eg youtube, using ipset tables.

[tohojo]

No, afraid not... qos-scripts and sqm-scripts will step on each other's toes if both are enabled.

[EricLuehrsen]

It would be long term but eventually QOS would get the best-practice results of SQM. So hfsc replaced by cake. Then scale various cake parameters proportiobally to allotted band width and DSCP squashed and replaced by MAC/application assignments. Or as an advanced option swith between htb+fqcodel or cake depending cake in kernel upstream... just speculation though.

[moeller0]

Hi Eric,

On Jan 23, 2017, at 17:14, Eric Luehrsen notifications@github.com wrote:

It would be long term but eventually QOS would get the best-practice results of SQM. So hfsc replaced by cake. Then scale various cake parameters proportiobally to allotted band width and DSCP squashed and replaced by MAC/application assignments. Or as an advanced option swith between htb+fqcodel or cake depending cake in kernel upstream... just speculation though.

I believe that cake explicitly does not want to expose the necessary level of detailed configurability to make it a “drop-in” replacement for hfsc on qos-scripts, especially in regards to a variable number of priority tiers above the exposed schemes like diffserv4 and friends as well as actually allowing to assign different bandwidth percentages to each band, let alone exposing the dscp to priority tier mapping. From that perspective I would not hold my breath for cake to replace hfsc in qos-scripts (and or sqm-scripts compleletly replacing qos-scrripts, as there will always be users who want/need more exposed configurability).

Best Regards

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[EricLuehrsen]

If cakes long term vision is limited as such, then cake will find it hard for upstream general acceptance. A service provider head end needs to provide classful throttling to each customer per tier purchased. The might also want to apply DSCP / port / or deep packet inspection classification under each customer. hfsc and htb with the rest of ipt/tc configuration allow for this and will get a lot more commercial support. not saying its bad or good. only this is the consequence of certain choices.

I believe that cake explicitly does not want to expose the necessary level of detailed configurability to make it a “drop-in” replacement for hfsc on qos-scripts, especially in regards to a variable number of priority tiers above the exposed schemes like diffserv4 and friends as well as actually allowing to assign different bandwidth percentages to each band,......

[moeller0]

Hi Eric,

On Jan 23, 2017, at 18:33, Eric Luehrsen notifications@github.com wrote:

If cakes long term vision is limited as such, then cake will find it hard for upstream general acceptance.

This might or might not be true; cake’s aim at being basically a sane defaults shaper for the rest of us might still be sufficient to merit upstreaming.

A service provider head end needs to provide classful throttling to each customer per tier purchased.

I am very doubtful that any service providers actually use HTB/HFSC/fq_codel but rather whatever shaping is available at their cisco/juniper/what ever access concentrators. At least my ISP uses a juniper shaper/throttle with undisclosed parameters. I would not be amazed if at least bigger ISPs also would aim for centralized shaping at a network level that might overwhelm cake (or rather a level in which cake is comptationaly too expensive). So not explicitly aiming for usability for ISPs might not be a merge-blocker (not attempting to even rfc it on lkml certainly is a bigger stumbling block so far).

The might also want to apply DSCP / port / or deep packet inspection classification under each customer. hfsc and htb with the rest of ipt/tc configuration allow for this and will get a lot more commercial support. not saying its bad or good. only this is the consequence of certain choices.

I believe that cake explicitly does not want to expose the necessary level of detailed configurability to make it a “drop-in” replacement for hfsc on qos-scripts, especially in regards to a variable number of priority tiers above the exposed schemes like diffserv4 and friends as well as actually allowing to assign different bandwidth percentages to each band,……

Exactly, from the man page in the section pririty queue parameters “Detailed  customisation of tin parameters is not provided.  The following presets perform all  necessary  tuning, relative  to  the  current shaper bandwidth and RTT settings.”. Personally I believe it would be a great addition to at least expose upper and lower bands per tier, as well as the DSCP to tier mapping in a generalized fashion, but I am merely an onlooker (with a strong opinion, not necessarily supported by knowledge) and only wanted to give my current interpretation of cake’s design goals as I understand them at least.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[braian87b]

@tohojo there is another way to accomplish this ? no solutions so far?

[moeller0]

Silly question, have you tried cake’s isolation options, especiallt “nat dual-srchost" on egress and “nat dual-dsthost” on ingress will at least give you a good approximation of fairness by internal IP addresses. So this will not allow you to throttle specific IPs or MACs, but it will at least not give any MAC the ability to completely hog a link with its flows. (Assuming the problematic internal hosts are not actively hostile in the sense of UDP floods and the like, this seems to work quite well). This certainly is not what you seem to want, but maybe it is good enough until you manage the solution you consider proper. Personally I can see a valid use-case for de-prioretizing specific IPs into the background class, sort of a way of making doubly sure bit-torrent will stay in the background (at least for egress)… but I assume that this would be far less than what you require…

Best Regards

On Jan 23, 2017, at 19:49, braian87b notifications@github.com wrote:

@tohojo there is another way to accomplish this ? no solutions so far?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chros73]

no solutions so far?

I ended up using gazambuja's suggestion, enabling another basic qdisc type of queue on the br-wifi interface (I wanted to limit upload/download speed capabilities of devices that use only wifi). Not very elegant but it does its job pretty well. The whole config looks like this:

config queue 'eth1'
        option qdisc_advanced '0'
        option enabled '1'
        option interface 'pppoe-wan'
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'cake'
        option script 'layer_cake.qos'
        option linklayer 'atm'
        option overhead '44'
        option upload '24500'
        option download '87500'

config queue
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'fq_codel'
        option linklayer 'none'
        option enabled '1'
        option interface 'br-wifi'
        option download '4400'
        option upload '54272'
        option script 'simplest.qos'
        option qdisc_advanced '1'
        option squash_ingress '0'
        option ingress_ecn 'NOECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '0'
        option squash_dscp '0'

[braian87b]

Yeah, that is great, you can even configure macaddress to use a specific gateway ip and set that ip in other device bridged to same wifi or eth but the problem resides when one wants to slowdown based on remote websites/IP's... if someone wants to slowdown facebook or youtube, etc that method does not apply... I have used successfully the patch for simple.qos but it chokes since some values are wrong... I think that this: http://serverfault.com/questions/350023/tc-ingress-policing-and-ifb-mirroring/386791#386791 could be adapted... using iptables to mark ip's from ipset and that way slowdown remote ip's... but don't know if that will interfere too in some way with the already running simple.qos from sqm-scripts. Currently I have two running queues on each mwan3 wan interface (wan and wan2) each one with their specific upload and download speed at 85% advertised speed (recommended values, or 90% tested speed). Thanks.

[moeller0]

I could be wrong, but I believe on an ifb you can not use iptables, you could tc hash table based filters. But if you actually are more concerned about external host fairness you could simply use dual-srchost on ingress and dual-dsthost on egress or try triple-isolate which seems to promise something inbetween internal and external host fairness. The nice thing about the dual- and the triple- option is that they will for each group still also maintain per-flow fairness so generally work well enough for most people to not bother any more with more detailled qos configuration. (That said, it currently is not clear to me whether triple-isolate works as intended, but that is because I do not fully understand how it is intended).

[braian87b]

@moeller0 Thanks for the advice, it sounds great, but Im afraid that I don't know how to do that, can you be so kind to point me with some link or code sample to achieve that?

I think that my need is a very common need, but just people forget about doing it and just go cold-turkey and block the access to content. I think that if we put the needed code to make this work somewhere visible will be broadly used.

[moeller0]

If your question is about trying cake, your best bet is to get LEDE 17.01 on your router and or install and build cake and tc-adv from source (see https://github.com/dtaht/sch_cake and https://github.com/dtaht/tc-adv) buiild it install it and then also install sqm-scripts on your computer. Again if you are wanting to deploy on a router, just install LEDE 17.01 (or current trunk on it), https://forum.turris.cz/t/how-to-use-the-cake-queue-management-system-on-the-turris-omnia/3103 while aimed at turris OS still contains all necessary pointers to configure sqm scripts with cake's isolation options. In the past the one thing users requested is per-internal-host fairness and thanks to cake sqm-scripts now can offer this. I believe that any much finer control is out of scope for sqm-scripts, but you could always use sqm-scripts machinery and just supply your own shaper.qos script in /usr/lib/sqm but you would be on your own in actually writing that. If you are talking about the tc hash functionality I can not offer any useful advise since I never got around playing with that.

[braian87b]

I have several routers around with OpenWrt 15.05.1 and some on production in home and office with sqm-scripts installed and configured with minimal basic and default settings (just the speeds are set), but my knowledge on tc and related currently ends there. Is LEDE 17.01 more advanced than OpenWrt 15.05.1 or it is the same ? I know that I can use MARK and CLASSIFY using iptables to set a mark/classid to packages, I already done that, but I feel lost with tc commands or with the different sqm algoritms/scripts/configs. Thanks.

[dtaht]

On 1/24/17 4:19 AM, braian87b wrote:

@moeller0 https://github.com/moeller0 Thanks for the advice, it sounds great, but Im afraid that I don't know how to do that, can you be so kind to point me with some link or code sample to achieve that?

I think that my need is a very common need, but just people forget about doing it and just go cold-turkey and block the access to content. I think that if we put the needed code to make this work somewhere visible will be broadly used.

This is actually a pretty clever idea. You could leverage the sqm configuration to pull the max out and give clients you don't like a percentage of that.

One way an anal admin (of which I am one) could do this is use the various rate limiting options in iptables matching against mac address.

I would not mind rate limiting unknown or IoT devices by default, assuming they could be recognised (via a pattern match on the mac),

This would only work for outbound.

or ... via the interface ("guest" gets a max of X percent) which could work for inbound and outbound....

Otherwise per-host fairness is more general and superior, and in either case I'd not put it into sqm but it's own module.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tohojo/sqm-scripts/issues/23#issuecomment-274788843, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGoilxcLS_kbzcwFOpL82wSR8t2XXV9ks5rVexXgaJpZM4HNqj4.

[EricLuehrsen]

If you use dhcp to white list "good" hosts from "less good" hosts, statically assign good and dynamicaly assign another, then you can use iptables natural connection across NAT between port and private ip. You can then packet limit wan-lan and lan-wan for an IP range. There the MAC part is dhcp...

• Eric

-------- Original message -------- From: Dave Täht notifications@github.com Date: 1/24/17 17:25 (GMT-05:00) To: tohojo/sqm-scripts sqm-scripts@noreply.github.com Cc: Eric Luehrsen ericluehrsen@hotmail.com, Mention mention@noreply.github.com Subject: Re: [tohojo/sqm-scripts] How to set download/upload limit per IP/MAC of device? (#23)

On 1/24/17 4:19 AM, braian87b wrote:

@moeller0 https://github.com/moeller0 Thanks for the advice, it sounds great, but Im afraid that I don't know how to do that, can you be so kind to point me with some link or code sample to achieve that?

I think that my need is a very common need, but just people forget about doing it and just go cold-turkey and block the access to content. I think that if we put the needed code to make this work somewhere visible will be broadly used.

This is actually a pretty clever idea. You could leverage the sqm configuration to pull the max out and give clients you don't like a percentage of that.

One way an anal admin (of which I am one) could do this is use the various rate limiting options in iptables matching against mac address.

I would not mind rate limiting unknown or IoT devices by default, assuming they could be recognised (via a pattern match on the mac),

This would only work for outbound.

or ... via the interface ("guest" gets a max of X percent) which could work for inbound and outbound....

Otherwise per-host fairness is more general and superior, and in either case I'd not put it into sqm but it's own module.

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tohojo/sqm-scripts/issues/23#issuecomment-274788843, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGoilxcLS_kbzcwFOpL82wSR8t2XXV9ks5rVexXgaJpZM4HNqj4.

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/tohojo/sqm-scripts/issues/23#issuecomment-274959116, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQSYYJxua5uIBPaTb3pqBpBXBGFktpP0ks5rVno0gaJpZM4HNqj4.

[braian87b]

@EricLuehrsen @dtaht Yeah, my next try will be do some experiments with packet limit in iptables, next I will keep trying other tc snippet's that I found (I have several browser tabs with info to read and try out)... I have plans to put computers in one network and other for cellphone and others devices on other network with less priority/bandwidth, and have 2 o 3 resctriction levels on computers, adv users will full access, common users with ads/malware websites restricted, and maybe a 3rd with just a whitelist, etc. Thanks.

[heri16]

Still no solution for bittorrent flows?

[moeller0]

Still no solution for bittorrent flows?

Well, part of the problem is that bit-torrent when using LEDBAT/µTP (see ) will keep pushing packets until they encounter 100ms of induced delay. But sqm will in almost all conditions manage to keep induced latency well below 100ms even under congestion, see https://perso.telecom-paristech.fr/drossi/paper/rossi13tma-b.pdf and http://caia.swin.edu.au/cv/ralsaadi/preprint/ralsaadi-lcn2017-preprint-aqm_on_ledbat-preprint-4oct17.pdf for a thorough discussion of the issue. Personally I would probably try to disable µTP since from my perspective 100ms induced latency just seems a bit too much to stomach, but then I do not torrent very much to begin with. I note that cake's dual-isolation modes can help to some degree to localize the "pain" to the IP address acting as torrent-endpoint, but as the papers show, LEDBAT will keep sending too aggressively and hence is likely to clog up the buffers of the upstream end of the bottleneck link, and there the induced delay will cause pain to all flows towards the bottleneck.

[dtaht]

moeller0 notifications@github.com writes:

Still no solution for bittorrent flows?

? I use torrent all the time. It's invisible with either fq_codel or cake with its default setup.

1) when dropping packets in under 100ms, it reverts to "reno" behavior, and a smaller initial window, so it competes with cubic and quic a little less hard.

2) Most torrent implementations limit you to 5 or so active connections - which is a lot less flows than you generate with web traffic.

3) Most torrent seeders are not running all that fast

To me, torrent's brain damaged 100ms back off is just completely ignorable, so long as you don't increase the defaults to hundreds of connections.

I used to set the QoS values to CS1 on my torrent client but that only worked for tcp, they forgot something (at least in transmission) to do it on utp. Never got around to fixing it. (it's in libutp somewhere, a single setsockopt is needed).

lastly, cake regulates "per host" fq, so a torrenting host can only take up 1/hosts amount of bandwidth, much better than fq_codel does, even if you make torrent open tons of connections.

...

sadly I see torrent use is also in decline, I've been trying to restore my copy of the quad version of wish you were here for a month now....

...

yes, libutp had a bug with pie. doesn't matter. it really doesn't.

Well, part of the problem is that bit-torrent when using LEDBAT/µTP (see ) will keep pushing packets until they encounter 100ms of induced delay. But sqm will in almost all conditions manage to keep induced latency well below 100ms even under congestion, see https://perso.telecom-paristech.fr/drossi/paper/rossi13tma-b.pdf and http://caia.swin.edu.au/cv/ralsaadi/preprint/ralsaadi-lcn2017-preprint-aqm_on_ledbat-preprint-4oct17.pdf for a thorough discussion of the issue. Personally I would probably try to disable µTP since from my perspective 100ms induced latency just seems a bit too much to stomach, but then I do not torrent very much to begin with. I note that cake's dual-isolation modes can help to some degree to localize the "pain" to the IP address acting as torrent-endpoint, but as the papers show, LEDBAT will keep sending too aggressively and hence is likely to clog up the buffers of the upstream end of the bottleneck link, and there the induced delay will cause pain to all flows towards the bottleneck.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.