Remove datasets with multiple versions

ESMValGroup / ESMValTool_sample_data

Samples of publicly available datasets for use with ESMValTool

Apache License 2.0

0 stars 0 forks source link

Remove datasets with multiple versions #12

Closed stefsmeets closed 3 years ago

stefsmeets commented 3 years ago

Fixes #5

I added a function that selects only the latest version of each dataset in the download script.

The following all have newer versions:

CMIP6/CMIP/CCCma/CanESM5/historical/r1i1p1f1/Amon/ta/gn/v20190306
CMIP6/CMIP/CCCma/CanESM5/historical/r1i1p1f1/day/ta/gn/v20190306
CMIP6/CMIP/NCC/NorCPM1/historical/r1i1p1f1/Amon/ta/gn/v20190914

stefsmeets commented 3 years ago

If it is a bug in the esgf downloader, then it would be better to implement the fix there, which is a bit out of scope for us here. I suggest merging this and keep this in mind for future development in a new issue.

bouweandela commented 3 years ago

If it is a bug in the esgf downloader, then it would be better to implement the fix there

Did you look into this? Is it a bug in esgf-pyclient or the ESGF API or do we somehow not understand how to use it correctly?

stefsmeets commented 3 years ago

Should we fix the download script to fix this instead?

bouweandela commented 3 years ago

Yes, that would be the easiest solution

stefsmeets commented 3 years ago

Hi @bouweandela , I added a function to select only the most recent version of each dataset. Could you have a look?