Introduce and leverage achieved rates ewma

[lynxthecat]

@bairhys any chance you could test this branch on your 4G (or is it 5G) connection? I am keen to know how switching to working with the achieved rate ewma (with alpha = 0.2) affects the performance of cake-autorate.

I think you could test simply by running:

sh setup.sh lynxthecat/cake-autorate achieved-rate-ewma

and to revert:

sh setup.sh lynxthecat/cake-autorate master

(or by manually swapping out cake-autorate.sh and defaults.sh).

[bairhys]

@lynxthecat I have 4G. I have just installed this branch. Easy to install. Looks like I need to make my prometheus exporter read a few extra log columns since it has stopped but that's ok.

root@OpenWrt:~/cake-autorate# sh setup.sh lynxthecat/cake-autorate achieved-rate-ewma
At least one instance of cake-autorate appears to be running - exiting
If you want to install a new version, first stop any running instance of cake-autorate
If you are sure that no instance of cake-autorate is running, delete the /var/run/cake-autorate directory
root@OpenWrt:~/cake-autorate# service cake-autorate stop
root@OpenWrt:~/cake-autorate# sh setup.sh lynxthecat/cake-autorate achieved-rate-ewma
Installing cake-autorate in /root/cake-autorate...
Previous configuration present - keep it? [Y/n] y
Using prior configuration

3.1.0-PRERELEASE successfully installed, but not yet running

Start the software manually with:
   cd /root/cake-autorate; ./cake-autorate.sh
Run as a service with:
   service cake-autorate enable; service cake-autorate start

root@OpenWrt:~/cake-autorate# service cake-autorate start

[lynxthecat]

Jury is out whether this offers an improvement and the optimal value of alpha (which affects how much smoothing the ewma does in respect of the achieved rate used by the shaper rate controller). There is a discussion on the OpenWrt forum about this change here. I'd really value your feedback from testing.

[lynxthecat]

To make the comparison fair, can you also please amend your defaults.sh to use:

high_load_thr=0.75

I've just force pushed an update to that original value.

[lynxthecat]

@rany2 would you be able to run a couple of runs on your connection through https://www.waveform.com/tools/bufferbloat and link results using this branch and also using master? I am keen to see how the ewma code affects things on your connection. The config values could remain the same from our last run.

[bairhys]

I haven't been following the conversation on this but I updated my high_load_thr=0.75. I also made a new branch for my prometheus script for the extra log columns added to this branch. Can see the new columns filtering the measured bandwidth:

[lynxthecat]

@moeller0 notice in the plot above that the decrements in the cake rate are always at points in time in which the ewma is actually a smidgen lower than the achieved rate?

[lynxthecat]

@bairhys any comment on how performance compares using the achieved rate ewma?

[bairhys]

I haven't done any back to back tests to see a difference yet. Plus I haven't read and understood the changes to know what to look for. But haven't had any issues, works at least as good as v3.0.1.

[moeller0]

On 26 July 2023 14:03:03 CEST, lynxthecat @.***> wrote:

@moeller0 notice in the plot above that the decrements in the cake rate are always at point sin which the ewma is actually a smidgen lower than the achieved rate?

This plot shows the shaper rate subsampled for times where we also have achieved rate data. I do not think it should be analysed to the level you seem to be applying.

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[lynxthecat]

Actually I think I was wrong again - the EWMA is always higher:

Still, the choices look appropriate to me - I mean using the EWMA of the achieved rate (orange circles) rather than the raw achieved rate for those drops seemed OK.

The difference would have been substantial at the third download drop.

I cross-referenced against the data at:

https://gist.githubusercontent.com/rany2/b6961fc913fb01eef42f77dec56d7026/raw/d3eeb7ed2e3737a11c3c79c1824782cbd34ac081/gistfile1.txt

once imported into excel to verify that I was looking at the right thing. Namely the shaper rate upon the drop (dl_high_bb or ul_high_bb) is set based on the achieved rate ewma at the point of time of the call of update_shaper_rate(), and following that rate change each DATA output line is output to the log file.

[moeller0]

As I said, once you use a low pass filter you will end up using smaller rate adjustments, which is OK if the achieved rate was 'too' low by pure chance and the true bottleneck rate either always was above the achieved rate or recovered quickly, but it is less so if there truly was a steep bottleneck rate drop, in which case using the gentler EWMA(achieved rate) will increase the time until the shaper rate is again below the true bottleneck rate which under load means longer and larger latency spikes... but it is exactly this situation where using the achieved rate over a fixed fraction of the last shaper rate promises to help temporal fidelity of our controller most...

Actually I think I was wrong again - the EWMA is always higher:

Still these plots do not show the actual times when the shaper rate was reduced so I would caution of interpreting these too deeply... this is not intended to diss these nice plots that are clearly useful, just not for precisely reasoning about temporal sequence at time of rate step-downs.