Roc already has unit and integration tests, but they, by design, have limitations:
they are run frequently during development, so they should not take long
they are run everywhere (on developer machine, on CI), so should not be sensitive to real-time (e.g. tests should pass even on loaded machine like CI VM worker)
they are yes/no tests, i.e. should either succeed or fail
We want to develop a new set of real-time tests:
running on real hardware on unloaded system
running long if needed
providing metrics, not just success or failure
testing various real-time aspects of the toolkit, for example latency, number of glitches, etc
These new tests will live in a separate repo: rt-tests.
Currently, the new repo has a build system, and a skeleton of one test that implements full loopback: writes audio stream to roc sender, obtains the stream from roc receiver, and runs a very basic check.
Now we need to develop a few actual tests:
service quality test (measure number of glitches to check how roc copes with packet losses and delays)
latency test (measure delay from sender to receiver)
stress test (check invariants under intense load)
and others
Please refer to rt-tests repo issues for more detailed description of each test:
Roc already has unit and integration tests, but they, by design, have limitations:
We want to develop a new set of real-time tests:
These new tests will live in a separate repo: rt-tests.
Currently, the new repo has a build system, and a skeleton of one test that implements full loopback: writes audio stream to roc sender, obtains the stream from roc receiver, and runs a very basic check.
Now we need to develop a few actual tests:
Please refer to rt-tests repo issues for more detailed description of each test:
rt-tests repo uses C++17, cmake, and Google Test.