Some utilities for comparison notebooks

[SarahAlidoost]

cc @Peter9192 and @sverhoeven: More path to ewatercycle system-related config file (grouped based on models):

• comparison_base_dir
• forcing_base_dir
• shape_dir
• aux_dir
• model_output_base_dir
• analysis_base_dir

Example usage would be: MODEL_OUTPUT_BASE_DIR = ewatercycle.config.CFG["model_output_base_dir"]["marrmot-m01"]

Generic util functions that can be used by most of the models:

• get_work_dir(absolute, relative, model_name) e.g. ewatercycle.config.CFG["output_dir"] / model_name_{time.strftime('%Y%m%d_%H%M%S')}"
• get_catchment_area(shapefile)
• generate_file_names(some_info)
• save_model_output(model_output, file_name) either time series at one location or spatial data at one time
• open_data_as_dataframe(file_name) e.g. get_precip_from_forcing(file_name) or get_simulated_streamflow(file_name)
• join_data_to_dataframe([simulated, observations, precipe])

Marrmot calibration module that can be used for three marrmot models:

• objective functions e.g. objective_function()
• running functions e.g. run_calibration()
• saving functions e.g. save_parameters() now it is based on yml but it can be csv.
• plotting functions e.g. plot_calibration()

lisflood specific that can be added to forcing module:

• generate_convex_hull_shapefiles(shape_file, model_mask)
• merge_forcing_files
• lisvap

I didn't look at hype, pcrglob and wflow.

[Peter9192]

As a scientist doing a comparison study using eWaterCycle, I'd like to have a nice location to store my DRS/paths, so I can easily find my files from within the notebooks.

We could make era5-comparison/utils/paths.py and do something like

from utils import paths
shapefile = paths.get_shapefile(catchment)

This would work for all paths that are specific to the comparison study but not generic enough for adding them to the main ewatercycle package. Can also be used for the work_dir and filenames

[Peter9192]

As a writer of the comparison paper, I want to refactor marrmot calibration code into a separte python module so I can easily reuse it, and the notebooks will still look clean

Since the BMI interface makes it relatively easy to do calibration "outside" of the main model code using a wide variety of calibration routines, we don't want to add it to the ewatercycle package (yet). However, if we find that some functions can be generic enough and useful, we might still do that in a later stage.

For now, let's make comparison.utils.calibrate.marrmot

[Peter9192]

As a user of lisflood, I want to have a lisvap specific module, so it doesn't distract too much from the main flow of the experiment in the notebook

Until we move lisvap to the ewatercycle.forcing module, we can make comparison.utils.lisvap

[Peter9192]

For PCRGlob, there are a couple of functions that might be generalizable as well or could at least be refactored into utils:

• create_clonemap, somehow this is part of the derivation of parameter sets, which is beyond the scope of the current work. But we can at least refactor it into a utility function.
• plot_catchment, which creates something like this:

• plot_forcing_data, which might as well be done inside the ESMValTool recipe (eventually).

[Peter9192]

For wflow the following candidates for refactoring:

• get_demfile_path (similar to shapefile)
• get_extents(mapfile)
• plot_catchments (similar to pcrglobwb)
• plot_forcing_data (similar to pcrglobwb)
• make_inifile (parameter set derivation, but might as well store it in a util)
• guess_outlet_gridpoint (even though we agreed to just specify it, we might still keep the code somewhere)
• load_streamflow & load_precipitation (I guess this can become one of the generic path utils Sarah already mentioned)
• select_catchment (or rather mask gridpoints outside the catchment): not sure if still needed
• plot_map (it's just a few lines of code, but perhaps it might help to increase the consistency across models