We have created this repo for various real-time integration tests for public API. See #1 for details.
This issue is for adding API benchmarks.
Currently we have two time-sensitive operations in API: roc_sender_write() and roc_receiver_read(). It would be useful to cover them with benchmarks and periodically check them for regressions. This is especially important for weaker CPUs like ARM.
The performance heavily depends on sender or receiver configuration, so we should check different combinations of all important parameters: resampler profile, FEC scheme, frame, packet, and FEC block sizes, packet interleaving.
Benchmarks should be done with automatic timing disabled (see API) because otherwise we will measure performance of sleep.
Benchmarks can print a table with results (measured performance for different combinations of parameters) to stdout.
We likely should employ Google Benchmark library. Then we'll also need to add it to CMake. Google Test is already present there.
Task
We have created this repo for various real-time integration tests for public API. See #1 for details.
This issue is for adding API benchmarks.
Currently we have two time-sensitive operations in API: roc_sender_write() and roc_receiver_read(). It would be useful to cover them with benchmarks and periodically check them for regressions. This is especially important for weaker CPUs like ARM.
The performance heavily depends on sender or receiver configuration, so we should check different combinations of all important parameters: resampler profile, FEC scheme, frame, packet, and FEC block sizes, packet interleaving.
Benchmarks should be done with automatic timing disabled (see API) because otherwise we will measure performance of sleep.
Benchmarks can print a table with results (measured performance for different combinations of parameters) to stdout.
We likely should employ Google Benchmark library. Then we'll also need to add it to CMake. Google Test is already present there.
Info
rt-tests repo currently has a skeleton of a test using public API: https://github.com/roc-streaming/rt-tests/blob/1212a60b0357c851282be3ec1cb1887e652ab22e/tests/test_service_quality.cpp
Public API is documented here: https://roc-streaming.org/toolkit/docs/api.html
FEC is documented here: https://roc-streaming.org/toolkit/docs/internals/fec.html
Another integration test using public API can be found here: https://github.com/roc-streaming/roc-toolkit/blob/master/src/tests/public_api/test_sender_receiver.cpp