ImperialCollegeLondon / pyrealm

Development of the pyrealm package, providing an integrated toolbox for modelling plant productivity, growth and demography using Python.
https://pyrealm.readthedocs.io/
MIT License
23 stars 9 forks source link

NA handling in subdaily models etc #70

Closed davidorme closed 7 months ago

davidorme commented 1 year ago

Describe the bug

The main PModel code is not bothered about np.nan values in inputs - there will be a gap in predictions but it all works. However, the memory_effect and fill_daily_to_subdaily methods in FastSlowPModel iterate over time series and get derailed by missing values.

There are various ways this could be handled - and these are most easily dealt with by removing np.nan values before fitting models. Then users can decide what to do rather than having baked in approaches. However, the methods should then look for np.nan in inputs (perhaps excluding leading and trailing np.nan) and raise a ValueError.

Expected behavior

davidorme commented 1 year ago

This is partly resolved by #78 closing #77, but this doesn't document the expectation that forcing variables should not have missing data.

MarionBWeinzierl commented 1 year ago

Could use None if using xarrays, but it's slow.

Helper function to check about NAs in the functions that care about NAs, but not in others?

Use a specific error check?

mondus commented 1 year ago

Good description of this here: https://www.residentmar.io/2016/06/12/null-and-missing-data-python.html

davidorme commented 7 months ago

This is now covered by #78 and also the more extensive documentation and handling of missing data arising from allowing partial start and end days with the subdaily model in #194.