NOAA-OWP / ngen

Next Generation Water Modeling Engine and Framework Prototype
Other
84 stars 63 forks source link

Combine all inputs into single realization file #495

Open jameshalgren opened 1 year ago

jameshalgren commented 1 year ago

The ngen CLI requires 5 inputs; optional inputs include a few more things. There might be some program execution bits that are still relevant on the command line, but for the most part, it seems like the realization file has the flexibility for all of the simulation-related inputs to be included.

One key advantage of such an approach would be that a validation checker (which doesn't exist yet) could verify all paths as real before execution from a single source file; if any paths are not valid, the execution can be stopped (or a warning issued) before any lengthy read/compute steps are taken. (This might be a place for a flag to "do it anyway, even though files are broken...")

Other advantages might be related to archiving/sharing of execution runs, such as might be done on HydroShare, Zenodo, or even in the canonical archives for NCEP. If a single file governs the run and all other inputs are publicly available (and wouldn't it be cool if they could be s3 URIs?!), then anyone can (theoretically) pull the realization file and repeat a given simulation.

Possible hangups might involve the need for some things to still hang out on the command line to support more flexible testing. E.g., if I want to try a serial and a parallel execution of the same hydrologic scenario, it's nice to be able to run ngen -r realization --serial and then to run ngen -r realization --parallel partition.file

jameshalgren commented 1 year ago

Probably a duplicate or extension of #159

hellkite500 commented 1 year ago

This is part of a larger discussion that is likely to ramp up in the near future as we look refactor the realization and input reading and processing.