Distributed modelling using HBVEC fails on PAVICS notebooks

[richardarsenault]

Description

Using notebook "/tutorial-notebooks/hydro/Distributed_hydrological_modelling.ipynb" works flawlessly with the default GR4JCN model setup. Changing to HBVEC (and changing the parameter set to reflect an HBVEC model) fails due to lake HRUs that are not accepted in HBVEC.

What I Did

1. Changed the model instantiation from GR4JCN to HBVEC;
2. Changed the parameter set to an HBVEC one.

The resulting block of code within the cell (cell 6) looks like this now:

model_config = HBVEC(
    params=[0.059,4.07,2.001,0.034,0.099,0.5,3.4,38.3,0.46,0.06,2.27,4.8,0.583,0.045,0.87,18.94,2.03,0.44,0.67,1.14,1.02],
    #params=[0.529, -3.396, 407.29, 1.072, 16.9, 0.947],
    StartDate=dt.datetime(1998, 1, 1),
    EndDate=dt.datetime(2020, 12, 31),
    ObservationData=[qobs],
    Gauge=meteo_forcing_stations,
    GlobalParameter={"AVG_ANNUAL_RUNOFF": 40.65},
    **rvh,
)

This fails with the following traceback:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[10], line 2
      1 # Prepare the model configuration
----> 2 model_config = HBVEC(
      3     params=[0.059,4.07,2.001,0.034,0.099,0.5,3.4,38.3,0.46,0.06,2.27,4.8,0.583,0.045,0.87,18.94,2.03,0.44,0.67,1.14,1.02],
      4     #params=[0.529, -3.396, 407.29, 1.072, 16.9, 0.947],
      5     StartDate=dt.datetime(1998, 1, 1),
      6     EndDate=dt.datetime(2020, 12, 31),
      7     ObservationData=[qobs],
      8     Gauge=meteo_forcing_stations,
      9     #GlobalParameter={"AVG_ANNUAL_RUNOFF": 40.65},
     10     **rvh,
     11 )
     13 # Run the model with the configuration we just built
     14 distributed_outputs = Emulator(model_config).run(overwrite=True)

File /opt/conda/envs/birdy/lib/python3.9/site-packages/ravenpy/config/emulators/hbvec.py:276, in HBVEC.__init__(self, **data)
    275 def __init__(self, **data):
--> 276     super().__init__(**data)
    278     if self.gauge:
    279         for gauge in self.gauge:

File /opt/conda/envs/birdy/lib/python3.9/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 11 validation errors for HBVEC
HRUs -> __root__ -> 45 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 65 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 66 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 68 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 84 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 85 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 86 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 96 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 146 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 152 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 164 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))

Expected Behavior:

I would expect the model to simply run and return a streamflow object.

[huard]

My understanding is that we have a few options here:

1. Modify the HBVEC emulator to add Lake HRUs. This involves adding a water land use class, soil profile, etc;
2. Filter out the lake HRUs before instantiating the model configuration (done explicitly by the user);
3. Filter out the lake HRUs within the model instantiation (invisible to the user, or with a warning);

I'm not sure it's a good idea to modify the emulator config and keep the same name (1.). Thoughts ?

[huard]

Example of code that would filter out lake hrus and warn the user:

class HRUs(rc.HRUs):
    """HRUs command for GR4J.

    Pydantic is able to automatically detect if an HRU is Land or Lake if `hru_type` is provided.
    """

    root: Sequence[LandHRU]

    @field_validator("root", mode="before")
    @classmethod
    def skip_non_land_hrus(cls, values):
        """Filter out non-land HRUs."""
        hrus = [hru for hru in values if hru.hru_type == "land"]
        if len(hrus) != len(values):
            warn("Only land HRUs are supported for this emulator. Other types of HRUs, such as lakes, were "
                             "filtered out.")
        return hrus

[huard]

I can apply the same logic to all emulators, using introspection to check what values of hru_type are recognized by the emulator. If the HRU values have a hru_type and it doesn't match the model, I would simply skip them from the config with a UserWarning.

[richardarsenault]

I think option 3 is the best way forward. Users know that when using the RavenPy distributed model builder, it comes with a whole bunch of hypotheses and simplifications. A user warning would suffice I think!

[lou-a]

In looking through the Raven manual v3.7 I found the following information that might be useful to treat lakes in HBV-EC:

Lakes in HBV By default, the :LakeStorage state variable is SURFACE_WATER. For HBV-EC or HBV light emulation this is modified to SOIL[2]. HBV treats all landscapes as if they have some presence of lakes, but does not explicitly treat LAKE HRUs. If you would like to revise HBV to properly handle precipitation on a lake, an additional command is required in the :HydrologicProcesses block: :Flush RAVEN_DEFAULT [lake storage SV] [SURFACE_WATER] :-->Conditional HRU_TYPE IS LAKE Without this, the precipitation in the ’lake’ storage unit may steadily increase because standard mechanisms for drainage (e.g., baseflow) are disabled for LAKE type HRUs.

In my opinion, skipping lake HRUs when they are small might be fine, but it could be problematic when they are meant to represent larger lakes. Alternatively (less preferable I think), since "HBV treats all landscapes as if they have some presence of lakes" using the :LakeStorage variable, could these lake HRUs be treated as land HRUs by the emulator?