ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

"synda search" does not return any reply due to variable_id error #107

Closed minomicetto closed 2 years ago

minomicetto commented 5 years ago

I have used synda 3.9 since 2017 without any problem, but recently I have noted that output of a simple command like

synda search pr cmip5

does not return any dataset anymore. The command returns some warnings

WARNING: 'pr' value has been associated with 'variable_id' facet. WARNING: 'pr' value has been associated with 'variable_id' facet. WARNING: 'pr' value has been associated with 'variable_id' facet.

The related HTTP Request built by synda (see request.get_url() in https://github.com/Prodiguer/synda/blob/4528d55b6422350ad05a0b6e4fb6c194449c37ac/sdt/bin/sdquicksearch.py#L99) is the following

http://esgf-data.dkrz.de/esg-search/search?variable_id=pr&fields=instance_id,id,variable,data_node,type,size&project=CMIP5&limit=100&distrib=true&type=Dataset&format=application%2Fsolr%2Bjson&offset=0

I am supposing the problem is due the name given to filter "variable_id", which should be "variable". In fact the query

http://esgf-data.dkrz.de/esg-search/search?variable=pr&fields=instance_id,id,variable,data_node,type,size&project=CMIP5&limit=100&distrib=true&type=Dataset&format=application%2Fsolr%2Bjson&offset=0

returns the correct output. If the behavior is correct, how should I use "synda search" to search dataset "by variable name" instead of "by variable id"?

AtefBN commented 5 years ago

Synda gets confused when trying to figure out through inference which key to use. Especially with the CMIP6 data being digested in solr bringing in new facet names. This is one case of that happening. You can check the param table in the db and you'll find both values variable and variable_id as names. Unfortunately, for the time being there's no work intended to be done on that module as we intend in the future to replace it altogether. So you might have to make your query more precise using key value pairs. Cheers!