neilh10
commented
1 month ago There has been a superhuman release of https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.35.0 thanks so much. I REALLY APPRECIATE the infrastructure that gets supported.
My question is, how should it be regression tested for all previous functionality ?
For open source software, regression testing is often something that the wider community can contribute. However this is only possible if its set up to be testable.
Practically speaking for my fork, "release1" stream based on the core (master), its going to take time to merge https://github.com/neilh10/ModularSensors/releases/tag/v0.34.1-abe and then I will be able to feedback any issues I find.
For testing community PRs on core code, it would be useful to have a model test setup - a Maylfy 1.1 with battery and optionally a variety of communication devices. For EASY regression testing, it must be possible to generate a new build, put it on the test system and then let it run for some time period for easy proof it works. Weekends are ideal times to leave a system running for an extended soak. The mayfly 1.1 has four basic sensors that can prove the reliability of the data flow.
Stability of a systems core software functions on an embedded uP is one of the most challenging aspects of an embedded system. Typically this is performed on a private sandbox, and consists of adding functionality and refactoring. The stability/function testing are called "regression testing", and can be both time consuming and boring. The opposite of the systems objective to be able easily extend a function, compile and commit.
There are many academic papers on the subject of adding functions and verifying "software correctness", promoting functionality through stages of stability - and there are many examples of embbeded at its essence there are three function 1) description of a change - including generating the code and generating a PR to pull into the code base 2) capturing the code and building in a traceable reliable manner (eg git source code, and build tools, and capturing libs)
3) proving that other functions aren't impacted regression testing including deploying built images to a farm of physical test setups automatically The git workflow supports branches and also a "develop" stream, for gaining confidence with new code.
Currently, it seems to me that SWRC, @SRGDamia1 critically performs item 2), It would be nice for that process to be supported with good data by users wanting 1) and prepared to do 3). Its critical that one person isn't response for system stability, and there is an easy way of distributing the ability to do regression testing, reporting, and collectively analyzing the results.
I currently have a version of this method on my own branch and would be happy to create a PR.
This would support a) say extras\ref_platform or examples\ref_platform - b) configuration (through local .ini) of the test setup, including defined all UUIDs in the .ini This could be modified later for git hub actions to automatically do a build, producing a .hex - that can then downloaded to a real system for core regression testing.