search

OpenCDSS / cdss-app-tstool-main

Colorado's Decision Support Systems (CDSS) TSTool application main program
GNU General Public License v3.0
1 stars 1 forks source link

Automated test report should indicate whether requirements are met #106

Open smalers opened 5 years ago

smalers commented 5 years ago

A number of automated tests in the summary report show failed tests because the datastore is not available, for example ReclamationHDB that is only available in client systems. This leads to a perception of more failures than actual. This could be resolved by defining and running test suites that exclude the problem tests. However, it would also be useful to add to tests a property like @requires datastore version that helps make sure a datastore is available for the test. The output report could then have a column indicating whether all requirements are met for the test. This could at least help sort through failed tests to know that something was not set up in the environment.

smalers commented 3 years ago

This could be handled by adding something like the following to @require annotations:

IfNotMet:WarnAndExit
IfNotMet:FailAndExit  (the default?)