Generalize input data - Githubissues

NCAR / container-dtc-nwp

End-to-end NWP systems in containers.

https://dtcenter.org/community-code/numerical-weather-prediction-nwp-containers

28 stars 13 forks source link

Generalize input data #54

Closed fossell closed 2 years ago

fossell commented 2 years ago

This PR makes an attempt to fix #37.

1) For #37, the following modifications were made in sha bb2757b:

an extra environmental variable (input_data) was added to set_env.ksh of each case where user can indicate which model input data is being used, e.g. "GFS".
The common run_wps.ksh script was updated to use this env var as the suffix to the Vtable that is used.
An additional if block was implemented to add the feature of looking first for an existing Vtable in the case scripts directory. If it exists, then use that since it is assumed to be custom to case needs. If it does not exist, then use the default Vtables provided with in the WPS.

Tests performed: Sandy on MacOS with main branch wps_wrf image built locally. Two tests, one with the existing Vtable.GFS present in the case directory and one without it present resulting in use of default Vtable.GFS from WPS. Both were successful through WPS steps (metgrid).

fossell commented 2 years ago

@michelleharrold - I put this PR in as a preliminary effort to address these two issues. But I think we need some discussion about how to finalize the changes and what would work best, and final testing. Let's chat and brainstorm when you have time!

michelleharrold commented 2 years ago

@fossell - awesome! I will take a look at the changes in the PR and maybe we can brainstorm tomorrow (I put something on the calendar for tomorrow but am flexible!).

fossell commented 2 years ago

I tested Sandy on my Mac through the Python step and all was successful. This was using images built locally from the main branch (i.e. updated software versions).

michelleharrold commented 2 years ago

I tested on my Mac -- built all images from scratch and ran end-to-end for snow and derecho cases. Everything worked as expected! I visually inspected output, and it all looks good. As a note, I tested with the Vtable in WPS v4.3 (not the ones in the case-specific directories). So it does appear that the old cases run with the lates Vtables!

fossell commented 2 years ago

Updated PR to include mods to address only #37. So input model_data path is still hard coded to look for gfs subdirectory.

fossell commented 2 years ago

@michelleharrold, @jwolff-ncar, @JohnHalleyGotway @mkavulich - Updated this PR to only include Vtable mods per discussion last meeting. My Sandy case test was successful. Looking for another approval and/or testing before merge.

fossell commented 2 years ago

This PR has been merged with main to include recent python updates. No conflicts or change to this PR, still awaiting approval.

fossell commented 2 years ago

@michelleharrold Thank you! I'm going to merge and get this one closed up.