NCAR / Topo

NCAR Global Model Topography Generation Software for Unstructured Grids
26 stars 16 forks source link

Add regression test to ensure bit-for-bit changes before making changes #14

Closed MiCurry closed 2 years ago

MiCurry commented 3 years ago

Before we make any major changes, creating a regression test to ensure we get the same results will be beneficial.

This issue serves as an area to discuss creating this regression test.

MiCurry commented 3 years ago

I think we should create a few 'control cases' that we can use to compare the tool going forward, which we could store on glade. We may also want to have their final namelistsassociated with them.

A good-enough regression test for this may be just taking the difference of a compare file with one that's generated with a new version. This could be done by hand using ncdiff or we could create a Python program that can be ran on glade.

We also may need to discuss, when we start to make major changes, if we expect the same settings to produce the same, or different results. At first, we can begin with comparing that namelists and output grids are bit-for-bit identical.

MiCurry commented 3 years ago

Somewhat along these lines, is there anyway to 'speed up' the topo program, even if it gives incorrect results? For the MPAS static interpolation we can subsample the GMTED data, which gives reduce resolution, but runs extremely quickly.

This feature is an easy way to make changes and just see if it 'runs'. Its an easy way to check to see if filenames are correct or if there are segmentation faults far into execution.

I have some 'hacky' ways that I do this, but does anyone have a recommendation for doing so in a different way? Maybe we could make a very course cube sphere for testing?

PeterHjortLauritzen commented 2 years ago

Done