NOAA-OWP / ngen

Next Generation Water Modeling Engine and Framework Prototype
Other
83 stars 62 forks source link

Loose Coupling #445

Open rburghol opened 1 year ago

rburghol commented 1 year ago

Can someone point to me where in the code it supports loose coupling with for example an upstream CSV representing flows out that should be inputs to the next segment.

Is there a simple technique, or some existing documentation if I had an existing model that gave me flows out of it and location that I wanted to connect to this model for testing purposes?

i've seen this issue on what I would describe as being tightly coupled, or as providing support for documenting linkages. See also #297

hellkite500 commented 1 year ago

@rburghol are you wanting to use your upstream CSV as input to a channel routing model? If so, you might want to look at t-route. This is the same channel routing embedded within ngen. Right now, there is no implemented support to do what you are asking in the model engine, though we might consider it in the future.

A little more clarification on your use case might help me to understand what that feature would look and what might be required to work towards an implementation.

rburghol commented 1 year ago

Thanks @hellkite500 -- your deduction about my use case is correct -- for 50% of my use case anyhow. I was interested in the ability of NGEN to accept upstream flows in a model-type agnostic manner, or, alternatively, to provide downstream flows in a model-agnostic manner.

My work involves running a handful of loosely coupled hydrologic models, and I am in the process of over-hauling our 15-year old model toolkit, and I was inspired by this from your README: "This is done by organizing model components along well-defined flow boundaries, and then implementing strict API’s to define the movement of water amongst these components."

So I was wondering if this API facilitated model-agnostic loose coupling (using something simple like CSVs or other common data transfer mechanism like hdf5 or even wdm) or if it was an internal only API, that the NGEN package uses to manage it's own sub-linkages.

Thanks for your consideration. I am reviewing t-route now, very interesting.

jameshalgren commented 1 year ago

Suggest [maybe] renaming this issue to "Create global warm state file" or "nested realizations" This mentions the idea of loose coupling specifically for routing-only simulations. That capability, generalized, could be useful for complex parallelization schemes and would require a robust global warm state file.

E.g., Hydrologic evolution in watersheds may be pleasingly parallelized if the concluding output is a time series of lateral inflows to the channel network. Hydraulic routing has much more significant strictures on parallelization due to the dendritic nature of flow, the non-linear effect of volume change at-a-point (i.e., we can't just do additive simulation), and the possibility for backwater effects propagating up (divergently, possibly) the channel network. This sets up a scenario where it might make sense to run the hydrology under a single ngen realization, write out a warm state file for routing initialization, then run the routing under another ngen realization with a different parallelization specified.

Similar situations could be imagined for any situation where loose might apply. Examples might include snow models with hydrology (runoff) models; short-term hydrology with groundwater models; routing models with ocean models; etc. Obviously, tight coupling might be the right approach in any of these cases depending on many things, but where appropriate, this would be a powerful testing tool and could be useful (as mentioned) in operational cases.