search

ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

Daemon stopping immediately #205

Open MariamZachariah opened 1 year ago

MariamZachariah commented 1 year ago

Hi, I've just set up synda via conda, and have been trying to download some cordex runs. However, the daemon stops immediately after i start running it. A colleague has used the same credentials on the same server, and successfully downloaded some data. I've compared our environments and we have the same versions installed. So not clear what the problem is. I've also run synda certificate renew and synda check-env, and both appears to have completed successfully.

Appreciate any help you could give me.

This is the last entry in the log file.

code=SDNETUTI-004,message=HTTP query failed (url=https://esg-dn1.nsc.liu.se/esgf-idp/openid/ClairBarnes,exception=HTTPSConnectionPool(host='esg-dn1.nsc.liu.se', port=443): Max retries exceeded with url: /esgf-idp/openid/ClairBarnes (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x15550818d940>, 'Connection to esg-dn1.nsc.liu.se timed out. (connect timeout=10)')),timeout=10) 2023-06-05 10:46:12,145 INFO SDDAEMON-034 Daemon stopped Daemon successfully started 2023-06-05 10:48:50,021 INFO SDDAEMON-001 Daemon starting ... INFO: Connected to /rds/general/user/mzachari/home/.synda/db/sdt.db 2023-06-05 10:48:50,022 INFO SDTSCHED-533 Connected to /rds/general/user/mzachari/home/.synda/db/sdt.db 2023-06-05 10:48:50,023 INFO SDTSCHED-993 Starting watchdog.. 2023-06-05 10:48:50,024 INFO SDFILDAO-200 get_files time is 0.000488, search select * from file where status=:status ORDER BY priority DESC, checksum with {'status': 'running'} FOLLOWING ERROR captured from ESGF openid Service...

code=SDNETUTI-004,message=HTTP query failed (url=https://esg-dn1.nsc.liu.se/esgf-idp/openid/ClairBarnes,exception=HTTPSConnectionPool(host='esg-dn1.nsc.liu.se', port=443): Max retries exceeded with url: /esgf-idp/openid/ClairBarnes (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x153606b06940>, 'Connection to esg-dn1.nsc.liu.se timed out. (connect timeout=10)')),timeout=10) 2023-06-05 10:56:26,052 INFO SDDAEMON-034 Daemon stopped Daemon successfully started

AtefBN commented 1 year ago

Hi @MariamZachariah, looks like the problem comes from the identity provider that is unavailable at the time of the request by synda. This could also explain why the behavior changed from your run and your colleague's. To be clear this is not solveable from synda's side, it is simply a client of the ESGF federation nodes. However I agree the silent stop/failure is a bit confusing. FYI, development on synda download tool has been halted as we moved to a new tool you can find here