404 Client Error: Not Found for url

[DoMatte]

Hi,

I am trying to download missing dataset while using using the ClimWIP recipe. It has worked in the past, but I have now this issues:

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://esgf-index1.ceda.ac.uk/esg-search ....

Note that I store my ESGF credential using keyring and that the missing data is currently on esgf I have uploaded my recipe and the debug.txt.

Any help would be greatly appreciated

Thanks

main_log_debug.txt recipe_climwip_brunner20esd_fixed.yml.txt

[valeriupredoi]

@DoMatte index1 has been retired and @bouweandela and myself have fixed the problem upstream in ESMValCore, so depending on your type of installation, you will have to either download the latest development main branch from ESMValCore, or re-create the conda env for ESMValTool so the new bugfix esmvalcore=2.7.1 (containing the fix) is picked up and used :+1:

[DoMatte]

Thanks @valeriupredoi , I somehow missed that. Just so I do not bother you with next changes, where should I have look for this? Also, would you be kind enough to guide to download the latest development main branch from ESMValCore?

And when you say re-create the conda env for ESMValTool so the new bugfix esmvalcore=2.7.1, is any os the method mentioned here work or you really have to used the one from source?

Thanks

[bouweandela]

There is no need to reinstall the software, you can just update the configuration. Create a file called ~/.esmvaltool/esgf-pyclient.yml if you don't have one yet and in it you can put the list of urls for the index nodes you would like to use:

search_connection:
  urls:
    - 'https://esgf.ceda.ac.uk/esg-search'
    - 'https://esgf-node.llnl.gov/esg-search'
    - 'https://esgf-data.dkrz.de/esg-search'
    - 'https://esgf-node.ipsl.upmc.fr/esg-search'
    - 'https://esg-dn1.nsc.liu.se/esg-search'
    - 'https://esgf.nci.org.au/esg-search'
    - 'https://esgf.nccs.nasa.gov/esg-search'
    - 'https://esgdata.gfdl.noaa.gov/esg-search'

The documentation for this configuration file is available here: https://docs.esmvaltool.org/projects/esmvalcore/en/latest/quickstart/configure.html#configuration-file

[DoMatte]

Thanks @bouweandela , It indeed works. I am not using ESMValCore: 2.6.0, any simple command to update it to 2.7.1?

[bouweandela]

I am not using ESMValCore: 2.6.0, any simple command to update it to 2.7.1?

We recommend doing a new installation instead of trying to update an existing one. This ensures you end up with a collection of packages that is compatible with each other and if you don't like the new version for some reason you can easily go back to the previous one. You can read all about it in our installation instructions, but basically installing the latest version is as simple as running

mamba create esmvaltool -n esmvaltool-v2.7
conda activate esmvaltool-v2.7