search

LibreQoE / LibreQoS

A Quality of Experience and Smart Queue Management system for ISPs. Leverage CAKE to improve network responsiveness, enforce bandwidth plans, and reduce bufferbloat.
https://libreqos.io/
GNU General Public License v2.0
464 stars 50 forks source link

Replace partial reload with full reload? #298

Closed rchac closed 1 year ago

rchac commented 1 year ago

There are two reload methods - full (runs at 4AM) and partial (runs every 30 mins). We had implemented partial reload in order to avoid packet loss when a full reload occurs, since that used to take a few seconds. Partial reload can handle simple changes (subscriber added, removed) but is not able to handle complex scenarios (change of network.json, change of subscriber AP, etc). It's intended to incrementally update the queues to reflect new customers added or removed that day, so that they don't overload an AP by being unshaped until a full reload occurs at 4AM.

Currently, it takes about 0.7325 milliseconds per subscriber to perform the full reload. Testing at 50ms intervals and repeatedly running full reloads, I cannot catch any actual loss of connectivity when the full reload occurs. If this is the case at scale, it would be nice to deprecate partial reload and just run the full reload hourly. Seeking feedback, especially from those who have observed any sort of packet loss when performing a full reload.

dtaht commented 1 year ago

I was under the impression we added this feature to please the 11k subs in NZ?

rchac commented 1 year ago

They had partial reload disabled last I checked. But that may have changed. @khandieyea do you all use the partial reload feature at all?

interduo commented 1 year ago

@rchac did You try to test full reload on 20k+ devices?

I think we could not use partial reload anymore. Code is complicated and main reason for doing partial reload is outdated (packet loss during reload). I didn't thought that it would be possible when v1.2 was released.

The only disadventage when using full reload is that the tc counters are being reset.

dtaht commented 1 year ago

Did we get feedback from anyone else?

vettamatt commented 1 year ago

We would benefit from frequent full reload, although I agree about packet loss. I saw some minor loss with around 3000 shapers, but we're nearing production so won't be able to test this further. We'll continue to use full reload.

dtaht commented 1 year ago

I am going to close this