Currently, it is not clear what the return values of some of the functions are and which exceptions they may throw.
As a direct result, the local_tests fail due to exceptions leaking down to the main.
My suggestion is:
Tests run as units that return whether they passed as a bool. They don't throw expected exceptions such as problems during the test itself.
Internal errors and failure reasons are logged.
My suggestion for the internal handling 1
Internal exceptions are of type SystemTestException and have one of the following 3 types all inheriting from the aforementioned class:
STBuildException when something during the build phase went wrong
STRunException when something during the actual test case went wrong
STValidateException when something during the validation went wrong
My suggestion for the internal handling 2
Internal exceptions are treated in place. Every function returns a success state as a bool and logs all reasons for a failure.
Currently, it is not clear what the return values of some of the functions are and which exceptions they may throw. As a direct result, the local_tests fail due to exceptions leaking down to the main.
My suggestion is:
Tests run as units that return whether they passed as a bool. They don't throw expected exceptions such as problems during the test itself. Internal errors and failure reasons are logged.
My suggestion for the internal handling 1
Internal exceptions are of type
SystemTestExceptionand have one of the following 3 types all inheriting from the aforementioned class:STBuildExceptionwhen something during the build phase went wrongSTRunExceptionwhen something during the actual test case went wrongSTValidateExceptionwhen something during the validation went wrongMy suggestion for the internal handling 2
Internal exceptions are treated in place. Every function returns a success state as a bool and logs all reasons for a failure.