mathworks / climatedatastore

Climate Data Store Toolbox for MATLAB
Other
37 stars 10 forks source link

Invalid ZIP file | ERA5 hourly data on pressure levels #1

Closed mammadha closed 2 years ago

mammadha commented 2 years ago

Hi, Thank you for developing such a useful toolbox.

I am trying to use it for downloading "ERA5 hourly data on pressure levels" (here).

It seems that the file is downloaded:

2022-03-11 20:51:06,377 INFO Welcome to the CDS 2022-03-11 20:51:06,379 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels 2022-03-11 20:51:06,499 INFO Request is completed 2022-03-11 20:51:06,501 INFO Downloading https://download-0015.copernicus-climate.eu/cache-compute-0015/cache/data5/adaptor.mars.internal-1647049186.17651-22468-7-e45254a6-156d-4f7b-9c03-7f29418ad10e.grib to C:\Users\moham\AppData\Local\Temp\tp33b706a1_fd39_4a79_9c08_fe94dd461ad0.zip (7K) 2022-03-11 20:51:06,863 INFO Download rate 19.5K/s

But then immediately, I get the following error:

Error using climateDataStoreDownload (line 62) Invalid ZIP file "C:\Users\moham\AppData\Local\Temp\tp33b706a1_fd39_4a79_9c08_fe94dd461ad0.zip".

I took a look at the source code, and it seems that options.format is forced to zip file, which is not available for this specific CDS dataset. I think it is where the problem arises, but I could not solve it.

I appreciate any help.

Sincerely, Mohammad

rpurser47 commented 2 years ago

Hi Mohammad,

Thanks for letting me know about this. I was able to reproduce with the following code:

datasetName ="reanalysis-era5-pressure-levels";
options.product_type = "reanalysis";
options.format = "grib";
options.year = "2020";
options.month = "01";
options.day = "01";
options.pressure_level = "1";
options.variable = "divergence";
options.time = "06:00";
[downloadedFilePaths,citation] = climateDataStoreDownload(datasetName,options);

I'll try to look at this today.

mammadha commented 2 years ago

Hi Mohammad,

Thanks for letting me know about this. I was able to reproduce with the following code:

datasetName ="reanalysis-era5-pressure-levels";
options.product_type = "reanalysis";
options.format = "grib";
options.year = "2020";
options.month = "01";
options.day = "01";
options.pressure_level = "1";
options.variable = "divergence";
options.time = "06:00";
[downloadedFilePaths,citation] = climateDataStoreDownload(datasetName,options);

I'll try to look at this today.

Thank you, Rob.

rpurser47 commented 2 years ago

OK, I've finally solved this problem. I did a major rewrite, and I should release a new version tomorrow. Note that many of these requests can take a long time to process. You can check on the status of your request by visiting the CDS request status page.

rpurser47 commented 2 years ago

release 2.0 is done. Let me know how it works for you!

mammadha commented 2 years ago

release 2.0 is done. Let me know how it works for you!

Thank you, Rob.