LibreQoE / LibreQoS

A Quality of Experience and Smart Queue Management system for ISPs. Leverage CAKE to improve network responsiveness, enforce bandwidth plans, and reduce bufferbloat.
https://libreqos.io/
GNU General Public License v2.0
440 stars 48 forks source link

Wrong rates - for default queues (flat design, autogenerated nodes) #118

Closed interduo closed 1 year ago

interduo commented 2 years ago
# Traffic from devices not specified in Shaper.csv will be rate limited by an HTB of this many Mbps
defaultClassCapacityDownloadMbps = 100
defaultClassCapacityUploadMbps = 100

LibreQoS.py doesn't generate proper rates for default queue.

It creates: tc class show dev ens16np0 | less

...
class htb 11:1 root rate 10Gbit ceil 10Gbit burst 0b cburst 0b 
class htb 11:2 parent 11:1 leaf f4bb: prio 5 rate 2500Mbit ceil 9999Mbit burst 1250b cburst 0b
...
rchac commented 2 years ago

defaultClassCapacityDownloadMbps and defaultClassCapacityUploadMbps are deprecated and no longer part of ispConfig.example.py. You need to use generatedPNDownloadMbps and generatedPNUploadMbps. Please see here.

interduo commented 2 years ago

Ok deprecated - so now we dont have opportunity to limit default class (users outside ShapedDevices.csv)?

rchac commented 2 years ago

Ok deprecated - so now we dont have opportunity to limit default class (users outside ShapedDevices.csv)?

Yes, you still can using generatedPNDownloadMbps and generatedPNUploadMbps

interduo commented 2 years ago

I would like to limit all unknown circuits (to 50M/50M). Can I just set those values to 50M?

rchac commented 2 years ago

That is a feature we can add. We could have the program add a generated circuit for every IP that is not already listed in ShapedDevices.csv, but that IS within some defined subnet of possible IPs.

The only downside here is it may add many filters, depending on how large the subnet of possible IPs is.

How big of a subnet of possible IPs do you have to work with? /20?

interduo commented 2 years ago

Maybe we could add a filter on whole interface and then add normal filters over it? (so the unknown circuits would have some filter attached and known circuits will also have filters)

And then we could just make some class for defaultClassCapacityDownloadMbps and defaultClassCapacityUploadMbps again? It was dropped due to code refactoring right?

rchac commented 2 years ago

I don't think that is possible. We have to define an individual qdisc for every IP involved, or else they will share traffic ceilings and enforcing 50Mbps per unknown circuit / IP for example would not work.

But we can definitely automatically create a class/qdisc for every possible IP. For /19 subnets and smaller it would be viable.

For example a user could have

automaticAddIPs = [100.64.0.0/20]
automaticAddIPsRateDL = 50
automaticAddIPsRateUL = 20

So for any IP within the range 100.64.0.0/20, if that IP is not already defined in ShapedDevices.csv, it will now be added with its own circuit at the default Rate

interduo commented 2 years ago

But we can definitely automatically create a class/qdisc for every possible IP. For /19 subnets and smaller it would be viable.

Some facts before implementing that:

  1. More filters (without trie support in xdp-cpumap-tc) means more CPU usage. This should be postponed to this feature implemented in xdp-cpumap-tc.
  2. We use 10.0.0.0/8 and 172.12.0.0/12 (divided in many subnets, all access interface got its own subnet) adding 17825790 (16,777,216 + 1,048,574) would extened first load time to few minutes.
  3. In that scenario we should use XDP_Redirect and then use nft and 4 level filtering.
dtaht commented 1 year ago

Is this still an issue?

interduo commented 1 year ago

Yes

rchac commented 1 year ago

@interduo are you still experiencing this issue in recent versions now that it supports arbitrary prefix length?

interduo commented 1 year ago

@rchac What limit should have Circuit without plan defined in ShapedDevices.csv?

rchac commented 1 year ago

@interduo If a circuit is not defined in ShapedDevices.csv it will not have speed limited. It will pass through.

interduo commented 1 year ago

If we assume that behavior is good then the queues are created propperly.