search

OpenCDSS / cdss-app-statecu-fortran-test

Automated tests for StateCU software.
0 stars 0 forks source link

Update to provide more summary information #8

Open smalers opened 3 years ago

smalers commented 3 years ago

The artifacts from the testing framework are useful. However, some improvement would make it easier to efficiently digest the results, such as the following:

  1. The output reports from the TSTool CompareTimeSeries command do not have comments indicating the number of differences and total number of time series. This information is available in the command status in TSTool, but it would be good to have in the report comments. The reason this is not currently done is that the information is not known until processing the reports and the header comments are currently added at the top.
  2. The TSTool CheckTimeSeries status message could provide more information such as the number of time series with differences at each tolerance level. Formatting this information is complicated by possible other filters such as precision.
  3. Currently, running the comparison creates a DateValue file in results for time series with differences. This streamlines accessing the data if a visualization is requested. However, I suspect that in datasets where a large number of time series is different, saving the DateValue file we not work well. The ts-diff.tstool command file may need to re-do the difference on only the requested matching time series. I'll revisit this as the StateMod tests are completed.
  4. Requiring the development environment may be a challenge for some that could use the script. Maybe also try running on Git Bash and Cygwin... but that is really just a different flavor of installing MinGW.