juseg / hyoga

Paleoglacier modelling framework
https://hyoga.io
GNU General Public License v3.0
8 stars 0 forks source link

Add CHELSA-ERA5 tiled climatologies. #97

Open juseg opened 1 month ago

juseg commented 1 month ago

Close #96.

juseg commented 1 month ago

This works. The data re-projection is blazing-fast in comparison to working with the geotiffs (as evidenced using a dask.diagnostic.ProgressBar(). However, the overall atmosphere() call time is barely faster. I think this is due to the large overhead of reading metadata from 864 netCDF tile files.

Now I assume CHELSA-W5E5 suffers the same problem. I need to think about that. Ideas for the way forward:

Probably a (yet another) separate issue. I like the third solution because (hypothetical, some-day) end-users would normally use data for all months on a given region. It would also pave the road towards smarter aggregators opening (or even downloading) only necessary files.

juseg commented 1 month ago

Benchmarks on an preparing an Alps domain dataset. Averages of 5 calls, using dask.diagnostics for reprojections time.

Total times still include re-projecting the global netCDF topography, which seems to take most of 6 secs in the last case.