NOAA-OWP / ngen

Next Generation Water Modeling Engine and Framework Prototype
Other
84 stars 63 forks source link

Enable user-settable output directory #374

Closed SnowHydrology closed 1 year ago

SnowHydrology commented 2 years ago

Current behavior

Nextgen currently writes catchment and nexus output to the main directory level with the format of cat-N.csv and nex-N_output.csv where N is the catchment and nexus id. This is coded as:

formulation->set_output_stream(feat_id+".csv");

and

nexus_outfiles[id].open("./"+id+"_output.csv", std::ios::trunc)

Expected behavior

Ideally, the user should be able to control the output directory and the filename. We should keep the current behavior as default, but enable user-defined values. Perhaps this can be done in a similar way to how the realization file handles forcing data.

"forcing": {
            "file_pattern": "{{id}}.csv",
            "path": "./forcing/",
            "provider": "CsvPerFeature"
}

Steps to replicate behavior (include URLs)

Links to where the output files are defined:

https://github.com/NOAA-OWP/ngen/blob/1b7988969a4a79ec9d205268191362a93995d35a/src/NGen.cpp#L264

https://github.com/NOAA-OWP/ngen/blob/1b7988969a4a79ec9d205268191362a93995d35a/src/core/HY_Features.cpp#L31

program-- commented 1 year ago

Based on this, we can implement something like the following in the realization configs:

{
  "global": {
    ...
    "outputs": {
      ["nexus"|"catchment"]: {
        // Output Directory, default is current working directory
        // from which the executable was called, i.e. "."
        "directory": "/tmp/outputs/",

        // Pattern for either Nexus or Catchment outputs,
        // defaults are "{{id=cat-N}}.csv" and "{{id=nex-N}}_output.csv"
        // for catchments and nexi, respectively. The IDs are direct
        // strings taken from the hydrofabric
        "file_pattern": "awesome-{{id}}.csv"
      }
    },
    ...
  },
  ...
}

Then, adding two member functions to realization::Formulation_Manager that require an ID parameter should make this fairly seamless:

// ngen/src/NGen.cpp:264
nexus_outfiles[id].open(manager->get_nexus_output_path(id), std::ios::trunc);
// ngen/src/core/HY_Features.cpp
formulation->set_output_stream(formulation->get_catchment_output_path(feat_id));

Probably just need to handle preceding and trailing path separators. Alternatively, directory and file_pattern could be merged together -- so that the JSON tree looks more like:

...
"nexus": "/tmp/outputs/awesome-{{id}}.csv",
"catchment": "/tmp/outputs/awesome-{{id}}.csv"
...