The #@require comment is now being used to check software and datastore versions to ensure that command files are run correctly. The following additional features (or a variation) are needed to implement robust testing.
Adding #@require filemodtime xxxxx > xxxxx comment will perform a check to make sure that the file modification time of one file is later than another. For example, when used with StateMod testing, a goal is to make sure that simulation output files are newer than the time on the response file (*.rsp) that controls the model run. Doing this check will ensure that any tests run on the output will be consistent with changes to the response file. The functionality will also provide useful feedback in production because modelers will be alerted to cases where multiple processing steps may not have been run as needed.
Adding the #@order before CommandFileName and #@order after CommandFileName comment will allow the testing framework to ensure that command files are run in the proper order. For example, one test may run the StateMod simulation, with a pass resulting from a successful full run (exit code 0). Other tests will depend on the results files, such as comparing time series from binary output files.
The above functionality working together will allow for robust testing to occur. Annotation comments can be added to command files to describe appropriate relationships and the testing framework will automatically ensure compliance from the comments. Technical work will involve some complexity, in particular handling relative paths and use of ${Property} notation in order to give flexibility in command files.
The
#@requirecomment is now being used to check software and datastore versions to ensure that command files are run correctly. The following additional features (or a variation) are needed to implement robust testing.Adding
#@require filemodtime xxxxx > xxxxxcomment will perform a check to make sure that the file modification time of one file is later than another. For example, when used with StateMod testing, a goal is to make sure that simulation output files are newer than the time on the response file (*.rsp) that controls the model run. Doing this check will ensure that any tests run on the output will be consistent with changes to the response file. The functionality will also provide useful feedback in production because modelers will be alerted to cases where multiple processing steps may not have been run as needed.Adding the
#@order before CommandFileNameand#@order after CommandFileNamecomment will allow the testing framework to ensure that command files are run in the proper order. For example, one test may run the StateMod simulation, with a pass resulting from a successful full run (exit code 0). Other tests will depend on the results files, such as comparing time series from binary output files.The above functionality working together will allow for robust testing to occur. Annotation comments can be added to command files to describe appropriate relationships and the testing framework will automatically ensure compliance from the comments. Technical work will involve some complexity, in particular handling relative paths and use of ${Property} notation in order to give flexibility in command files.