search

ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

Wget timeout config in source code #160

Closed AtefBN closed 3 years ago

AtefBN commented 4 years ago

wget timeout in daemon mode at the moment is encoded within the source in sdconts (line 150) This type of configuration needs to be outside in sdt.conf ideally to ensure users can change the behavior without changing the code.

hanjunkim0617 commented 3 years ago

Hello. My name is Hanjun Kim, who is using the Synda for downloading the large data from ESGF. Thank you for making such nice program.

During using the function "synda install", I found that some data is not downloaded but stays in the running queue for about 20 minutes before going to error queue. Is there any method to shorten this time? I think this is related to the "wget timeout in daemon mode" so upload the question here. I'm using Synda version 3.15 with Conda installation. In addition, I wonder if I change the options in bin/sdget.sh, it affects the downloading by "synda install" or not. (I have wget_timeout option in bin/sdget.sh, so does it affect the "synda install"?)

Thank you, Hanjun

pjournou-ipsl commented 3 years ago

A enhancement in synda version 3.15 allows you to update the timeout value that is passed as an argument when synda calls the wget HTTP client. This entry is located in the sdt.conf file, in the [download] section. Its name is async_http_timeout. By default, its value is 120 (that means 120 seconds). The value may be set lower if necessary.

hanjunkim0617 commented 3 years ago

Thank you for answering! I changed the option.