hydro not compatible with normalize_using_yearly option

davide-f commented 2 years ago

Description

The hydro feature, which is based on runoff function, cannot successfully exploit the normalize_using_yearly option of runoff. When using the pypsa-africa package, which relies on the hydro function, with the above-mentioned option (branch), the following error arises

error   File "/home/davidef/miniconda3/envs/pypsa-africa/lib/python3.10/site-packages/xarray/core/dataarray.py", line 727, in _getitem_coord
    var = self._coords[key]
KeyError: 'countries'

Expected Behavior

I'd expect the procedure to be able to normalize the inflows to matche the desired values.

Actual Behavior

The error mentioned above arises

Error Message

error   File "/home/davidef/miniconda3/envs/pypsa-africa/lib/python3.10/site-packages/xarray/core/dataarray.py", line 727, in _getitem_coord
    var = self._coords[key]
KeyError: 'countries'

davide-f commented 2 years ago

As additional comment, I think both hydro and runoff functions may be slightly revised to account for partially generic indexing. The above error is related to the fact that the runoff function and the normalization function is calibrated to be used in pypsa-eur having as input the countries shapes, with "countries" as column value. With hydro, time series for a series of location are drawn, though the column value is no more "countries" (it becomes "hid"), and that's the cause.

Personally, I think it could be nice to:

generalize the runoff function and probably the "simplest" way may be to simply drop the reindex function that leads to that issue. That reindexing may be done externally of that function
The normalization may account for more complex approaches: let's say that we have 3 known installed hydro ppls in a country and 5 buses (2 may have extendable hydro ppls). we may want to scale each runoff time serie of the 5 buses using the a coefficient tailored so as to match the known energy production of the 3 hydro ppls in the 3 buses where they are installed.

What do you think?

fneum commented 1 year ago

I think this is closed through #278.

PyPSA / atlite