This is so it is easier to figure out what solutions are potentially failing if they are upgraded.
I have also re-worked some of the setup scripts so that much less installing is needed. The checks run much faster for solutions that are installed via pip. If a solution needs to be compiled it still takes time. This rework puts way less stress on the GitHub machines, however, so iteration is much easier.
report generation still is not tested, but I do test that the logs generated by each solution can be properly parsed and the output is expected.
This is so it is easier to figure out what solutions are potentially failing if they are upgraded.
I have also re-worked some of the setup scripts so that much less installing is needed. The checks run much faster for solutions that are installed via pip. If a solution needs to be compiled it still takes time. This rework puts way less stress on the GitHub machines, however, so iteration is much easier.
report generation still is not tested, but I do test that the logs generated by each solution can be properly parsed and the output is expected.