Closed gulcing closed 1 month ago
valeriupredoi
commented
1 month ago @gulcing many thanks for the sight of this! Could I pls ask what data you are trying to download? I'll have a closer look at the login mechanism, but for now let's be sure that data actually exists on ESGF :+1: :beer:
sloosvel
commented
1 month ago The same thing happened to me. I just wanted to point out that you don't actually need credentials to download data from the ESGF, so if you remove your file or you comment all it's content it will still find the data and download it.
gulcing
commented
1 month ago @gulcing many thanks for the sight of this! Could I pls ask what data you are trying to download? I'll have a closer look at the login mechanism, but for now let's be sure that data actually exists on ESGF 👍 🍺
Thank you! I was looking for these data:
CMIP6.CMIP.EC-Earth-Consortium.EC-Earth3-AerChem.historical.r1i1p1f1.AERmonZ.ho2.gn
CMIP6.CMIP.EC-Earth-Consortium.EC-Earth3-AerChem.historical.r1i1p1f1.AERmonZ.oh.gn
I could download them, but I just wanted to highlight that the login is not working
valeriupredoi
commented
1 month ago good stuff, @sloosvel and @gulcing, cheers :beer: Indeed, no need to use the creds login file, unless for datasets that really do need a login (and I don't remember any more of those, used to be MIROC and a few others); but a good point nonetheless that the current config is not working well. I'll ping @bouweandela when back from hols see what we can do about it. @gulcing - the reason why I was asking what data you're after was that now a lot of CMIP5 is actually getting retired off ESGF, with planned retirement of the entire CMIP3 and CMIP5 pool very soon!
bouweandela
commented
1 month ago Logging in is no longer needed to download data from ESGF, so you can skip this part of the configuration. The login is not working in your case because the OpenID server at DKRZ appears to be no longer available. The same thing happened some time ago with the OpenID server at LLNL. I've asked on the ESGF mailinglist if the plan is to stop all OpenID login servers. If that is the case, we could remove any code and documentation related to logging in.
bouweandela
commented
1 month ago Here is the response I got:
The plan is to move completely to OpenID Connect and OAuth 2.0. We are working with the Globus team to integrate these. The plan is to have two synchronised identity proxies – a Globus one and a European one. At CEDA, we have been running a test service for some time now which is based on Keycloak. In time this will be replaced by the European proxy.
In answer to your question, I would expect that all legacy OpenID 2.0 services will be retired.
So we can remove the current code and documentation for logging in.
To download data (that are not available in Levante DKRZ and I had ESGF account), I created the esgf_pyclient.yml file as explained in https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#configuration-file, I realized that the ESGF Data DKRZ website had changed, and OpenID login is not possible anymore. The option for sign-in on new website is not working. Therefore, esgf_pyclient.yml is not working as well.