search

OpenCDSS / cdss-app-statecu-fortran-test

Automated tests for StateCU software.
0 stars 0 forks source link

Summary of 14.0.0 testing #6

Open smalers opened 3 years ago

smalers commented 3 years ago

I have created a README in the repository summarizing 14.0.0 comparison with 13.10. This page can be updated over time to record software version testing.

https://github.com/OpenCDSS/cdss-app-statecu-fortran-test/blob/master/test-results-doc/README.md

Below is an example of larger differences for the rg2012 scenario.

image

Below is an example from the rg2012_FactorSoUMeter scenario.

image

Below is the bit difference for SP2016:

image

It seems like maybe the latest version is handling input differently in some cases?

smalers commented 2 years ago

I updated the test results for StateCU 14.0.1, which fixed the problem of BD1 files not being created when isuply=0. I also reran the South Platte and updated the numbers. I may not have properly recorded those results. The South Platte comparison has more and larger differences than other datasets.

https://github.com/OpenCDSS/cdss-app-statecu-fortran-test/blob/master/test-results-doc/README.md

Automated testing would benefit from smaller datasets to verify calculations, similar to StateMod examples. Then it is would be possible to confirm model results for each feature.

This issue can be closed by the State when they are satisfied with the results.

Attached are the summaries of South Platte dataset differences. SP2016_Restricted-summary-differences.txt SP2016-summary-differences.txt rg2012_FactorSoUMeter-summary-differences.txt