EcoExtreML / zampy

Tool for downloading Land Surface Model input data
https://zampy.readthedocs.io/
Apache License 2.0
1 stars 0 forks source link

Optimize downloading process from CDS #22

Open geek-yang opened 1 year ago

geek-yang commented 1 year ago

In the current implementation, downloading request to CDS is raised one by one. This can be very inefficient, especially for requests per month (e.g. download era5 hourly surface_pressure from 2010 Jan to 2010 Dec, 12 requests will be made per month, in this case the queuing time is always more than the downloading time).

In era5cli, several download requests will be made and queued parallelly. We may consider support parallel request in the future to improve the efficiency.

PS: thanks @BSchilperoort for spotting this and made the suggestion.