search

LibreQoE / LibreQoS

A Quality of Experience and Smart Queue Management system for ISPs. Leverage CAKE to improve network responsiveness, enforce bandwidth plans, and reduce bufferbloat.
https://libreqos.io/
GNU General Public License v2.0
421 stars 47 forks source link

Much more elaborate test suite #153

Open dtaht opened 1 year ago

dtaht commented 1 year ago

Leveraging the resources we've garnered via #151 producing a repeatable test suite for comparisons of not just our stuff, but that of others, would be nice. I'm not going to lay out too much of what that looks like here as yet...

dtaht commented 1 year ago

https://www.linkedin.com/feed/update/urn:li:activity:6996837189460328449/ CI bootcamp

thebracket commented 1 year ago

I absolutely agree - CI and a test suite would be awesome. It's also a really big job, so it may take a while. There are some parts of the xdp code that I'm not sure how to unit test (I don't know how to run unit-tests on the BPF side when you have moving parts that require data transit from ingress XDP to egress TC BPF code. It's probably possible, I just haven't figured it out, yet). So on that side, some integration testing to quickly pull the latest XDP code and run some tests against it will be required. It looks like the orchestration system can do it, it'll take me a while to figure out exactly how to constrain the various moving parts.

One thing that's really handy (I use it all the time in my Rust libraries) is to have a standard benchmark and persistent test results. REALLY handy to see that not just "the code works", but "performance has changed by X%".

On Mon, Nov 14, 2022 at 4:25 AM Dave Täht @.***> wrote:

https://www.linkedin.com/feed/update/urn:li:activity:6996837189460328449/ CI bootcamp

— Reply to this email directly, view it on GitHub https://github.com/LibreQoE/LibreQoS/issues/153#issuecomment-1313440442, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRU43223ALC3MEFWPVBJJLWIIHSVANCNFSM6AAAAAAR7DUBA4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

dtaht commented 1 year ago

Maybe gathering requirements is a thing for another form of github, so we can all edit one page?

Traffic generators:

https://www.spirent.com/products/testcenter-hardware

https://www.candelatech.com/datasheet_fire.php

Correctness tests

https://www.iwl.com/network-protocol-testing

dtaht commented 1 year ago

I added some stuff for simulation, and the CI is a great start (tho I would like it to retain artifacts and have those get installed into the testbed automatically). I'm not sure how to use checkboxes. Still open here are:

[ ] Unit testing stuff (I'd wanted unit tests, for example, of structure size) [ ] Test suite (notably of 10k ips)

dtaht commented 1 year ago

What we have now, is rather hacky. I have a string of shell scripts, mostly, that help manage the testbed. Trying to find the right methods AND things to emulate has been a problem, and I diverted to writing out some ideas in C rather than any other language, just to better express myself for a while. I have an increasing grip on what I would like to do (containerizing a lot), and what we have has been good enough for v1.4 so far... mostly.