search

ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

splitting entries with newlines silently ignored #142

Open davidedelvento opened 4 years ago

davidedelvento commented 4 years ago

Installing a selection file such as:

project=CMIP6
experiment_id=ssp370
source_id=ACCESS-CM2 ACCESS-ESM1-5 AWI-CM-1-1-MR BCC-CSM2-MR BCC-ESM1
CAMS-CSM1-0 CNRM-CM6-1 CNRM-CM6-1-HR CanESM5 CanESM5-CanOE EC-Earth3
EC-Earth3-Veg FGOALS-f3-L FGOALS-g3 GFDL-ESM4 INM-CM4-8 INM-CM5-0
IPSL-CM6A-LR MIROC-ES2L MIROC6 MPI-ESM-1-2-HAM MPI-ESM1-2-HR,
MPI-ESM1-2-LR MRI-ESM2-0 NorESM2-LM NorESM2-MM
latest=True
replica=True
variable=tas siconc siextentn siearean areacell
frequency=mon

seems to work and files are downloaded. Only later the naive user discovers that many source_ids are ignored. The correct syntax is

project=CMIP6
experiment_id=ssp370
source_id=ACCESS-CM2 ACCESS-ESM1-5 AWI-CM-1-1-MR BCC-CSM2-MR BCC-ESM1 CAMS-CSM1-0 CNRM-CM6-1 CNRM-CM6-1-HR CanESM5 CanESM5-CanOE EC-Earth3 EC-Earth3-Veg FGOALS-f3-L FGOALS-g3 GFDL-ESM4 INM-CM4-8 INM-CM5-0 IPSL-CM6A-LR MIROC-ES2L MIROC6 MPI-ESM-1-2-HAM MPI-ESM1-2-HR MPI-ESM1-2-LR MRI-ESM2-0 NorESM2-LM NorESM2-MM
latest=True
replica=True
variable=tas siconc siextentn siearean areacell
frequency=mon

My gripe is that the first option should work, it's really not that hard to parse such a thing in python even without an helper library (even easier with a library).

If other things have priority and so parsing needs to be as it is.... my larger grief is that synda should at the very least spit out a warning (if not outright error) when the first selection file is installed.