Originally posted by **dhebbeker** January 18, 2024
We automatically run tests using the "native" build environment. The native build environment is the test environment available on standard desktop environments and on the continuous integration server.
Some of our code has *dependencies* to 3rd party libraries/code. And some of those libraries may not be compatible to the native build environment.
It is not necessary to test 3rd party libraries themselves.
It must be noted that the build management tool currently used (PlatformIO) does not support fine tuning the selection of sources to be compiled for test execution via the projects' build configuration (`platformio.ini`). Libraries can be partially compiled using a [`library.json`](https://docs.platformio.org/en/latest/manifests/library-json/index.html) file to filter files depending on the target build system. Apart from this libraries are build as a whole.
Mocking dependencies is technically possible but not effortless. Libraries such as [ArduinoFake][] can simplify mocking. But currently we use extensions from the Espressif Arduino framework. Those are not covered by ArduinoFake.
The used libraries do not (always) provide interfaces which are compatible to the test environment. That means, in general, that one needs to implement stubs for the 3rd party dependencies.
A possibility to avoid stubbing would be to avoid compiling code for tests which has dependencies to 3rd party code. The code which is then not tested on the native test environment should be kept to a minimum. *These are the 3rd party library adapters.* This code should be as simple as possible to reduce the risk of introducing errors. Those adapters are then excluded from testing using the native build environment.
[ArduinoFake]: https://github.com/FabioBatSilva/ArduinoFake/ "a simple mocking framework for Arduino"
I conclude these are the possibilities:
1. Exclude (private) libraries (especially the 3rd party libraries package) from tests which depend on libraries incompatible to the native build environment. Easiest solution; least code coverage.
2. Define `library.json` files for the (private) libraries to selectively exclude source files from testing which are incompatible to the build environment.
3. Attempt to mock and stub dependencies which can not be compiled using the native build environment. Highest code coverage; highest effort.
What are your thoughts on this? @FiveTeethless do you have any preferences?
What to do
[x] define how to deal with 3rd party dependencies while testing
[x] describe the requirements for the granularity of automated tests
Discussed in https://github.com/Task-Tracker-Systems/Task-Tracker-Device/discussions/100
What to do