NOAA-OWP / ngen

Next Generation Water Modeling Engine and Framework Prototype
Other
83 stars 62 forks source link

Allow "joining" of additional tables in the GeoPKG hydrofabric to get additional attributes #555

Open mattw-nws opened 1 year ago

mattw-nws commented 1 year ago

After #522 and #544, also add "joining" of additional attribute tables in the GeoPKG to divides/nexus tables. This will allow e.g. model-specific parameter tables to be added to the GeoPKG as a way of parameterizing models. Such tables should be expected to be 1-to-zero-or-1 mappings keyed on the feature ID.

Initially, a hardcoded list of expected tables may be fine. Eventually some method (discretization information for the layer in realization JSON) should exist to indicate table names to be included. Because of the order of processing, waiting to read the BMI model information from the realization will probably be too late a place to decided to bring the data in.

Current behavior

Only the nexus and divides(catchments) tables are read into the internal PropertyTree structure for the hydrofabric.

Expected behavior

Additional tables should be able to be read and their attributes included and passed to model modules as described in #544.

Steps to replicate behavior (include URLs)

1.

Screenshots

mattw-nws commented 1 year ago

This issue may take a different direction. Hydrofabric files are currently planned to include Parquet "sidecars" with the data in these tables instead of tables within the gpkg... so, either we will need support for Parquet in operations (not a given) or we will have to preprocess the parquet files and write the data into the GPKG, at least for ops... and if we do that, there's not really a reason to not just add the columns onto the features instead of adding tables that would have to be joined. So while we may still want to do this, it will probably be a lower priority.