search

OpenWaterFoundation / cdss-app-snodas-tools

Colorado's Decision Support Systems (CDSS) Snow Data Assimilation System (SNODAS) Tools
8 stars 4 forks source link

Write script to compare existing system output with new system #28

Open smalers opened 3 years ago

smalers commented 3 years ago

The 2021 snow season is over and we can now do the server update. We need a way to compare the old server results with the new code. Artifacts that could be compared:

  1. TIF images:
    1. Downside is there always seem to be some differences (mainly on the edges?).
    2. Can do some visual comparisons and write Markdown notes on the results.
  2. TSTool plot images
    1. It is difficult to compare PNGs other than binary different although visual inspection is good.
    2. Can do some visual comparisons and write Markdown notes on the results.
  3. CSV "ByDate" and "ByBasin" files:
    1. Can be automated with TSTool CompareTables and CompareTimeSeries commands.

The comparison probably does not need to be exhaustive but need to pick some dates in several years. The following could be done...

Use the GCP command line tools. See the following scripts, which upload TSTool installer and creates the index on the GCP site for examples of gsutil use. The gsutil (or whatever is needed) script can be used to retrieve files from SNODAS GCP and compare to local file, for example using TSTool. This can be run for specific dates.

Maybe write one script to automate download of GCP files to local files. Then use a TSTool command file with some commands like the SNODAS processing script. Instead of creating images, read the CSV files into time series and use the compare commands.

I recommend creating a folder in the repo to do the comparison to save scripts and TSTool command files. The output artifacts can be gitignored. Also save the Markdown with notes comparing. Then we have a record of what we did to compare.