Open juseg opened 1 month ago
This works. The data re-projection is blazing-fast in comparison to working with the geotiffs (as evidenced using a dask.diagnostic.ProgressBar()
. However, the overall atmosphere()
call time is barely faster. I think this is due to the large overhead of reading metadata from 864 netCDF tile files.
Now I assume CHELSA-W5E5 suffers the same problem. I need to think about that. Ideas for the way forward:
Probably a (yet another) separate issue. I like the third solution because (hypothetical, some-day) end-users would normally use data for all months on a given region. It would also pave the road towards smarter aggregators opening (or even downloading) only necessary files.
Benchmarks on an preparing an Alps domain dataset. Averages of 5 calls, using dask.diagnostics
for reprojections time.
'chelsa'
12 global geotiffs: reproj 2 x 5 secs, total 17 secs.'cw5e5'
864 monthly tiles: reproj 2 x 120 ms, total 9 secs.'cera5'
72 yearlong tiles: reproj 2 x 300ms, total 6 secs.Total times still include re-projecting the global netCDF topography, which seems to take most of 6 secs in the last case.
CERA5TiledAggregator
class.'cera5'
atmospheric data source.'cera5'
elevation data source.'cw5e5'
.whatsnew.rst
.Close #96.