pyet-org / pyet

:sunflower: pyet is a Python package to estimate reference and potential evaporation.
https://pyet.readthedocs.io/
MIT License
130 stars 34 forks source link

Speeding up calculations #46

Open JannisHoch opened 1 year ago

JannisHoch commented 1 year ago

On my Mac with 16GB RAM and M1 chip, processing roughly half a year of daily at 0.1 degree resolution and global scale** took me 30+ min*** for the Oudin method, accoring to % time in jupyter notebook. It used around 50 % of my CPU and 20 % of my memory, according to htop.

For methods requiring more input data, the processing time will likely increase.

Maybe worthwile looking into ways to speed up the calculations, such as numba, if that's compatible with xarray?

* so yes, using gridded data. And admittedly, it also contained the ocean surface. In total, 18003600 cells had to be processed per time step.

*** I stopped it then.

mvremec commented 1 year ago

Hi Jannis,

this is indeed a good point! Thanks! I will look into this and see if we can use our unvectorized functions or if we need to vectorize them. xarray.apply_ufunc might help.

Cheers, Matevz

To do: