Closed Irbis01 closed 2 years ago
Which wget2 version?
Wget2 1.99.1 on Ubuntu successfully downloads without getting stuck.
...
HTTP response 200 OK [http://f4se.silverlock.org/archive/f4se_0_06_21.7z]
Downloaded: 86 files, 29.29M bytes, 1 redirects, 0 errors
Most probably, it was a server issue.
PS - BTW, this makes me think we should show time taken to complete the wget2 command with average download rate maybe. Something like "Downloaded: 86 files, 29.29M bytes, 1 redirects, 0 errors, in m minutes, with average download speed: s Kbps"
OS: Ubuntu 20.4
command:
wget2 --version
result:
GNU Wget2 1.99.1 - multithreaded metalink/file/website downloader
+digest +https +ssl/gnutls +ipv6 +iri +large-file +nls -ntlm -opie +psl +iconv +idn2 +zlib +lzma +brotlidec +bzip2 +http2 +gpgm
I compare the number of downloaded files with the already downloaded site and their number is the same, but wget2 does not complete its work. freezes occur irregularly. This is very similar to the absence of a response from the server, but all the files are already downloaded
@Irbis01 Run your command with -d
to get the debug output. Copy and paste the complete o/p in a text file and upload that file here.
Also, do mention exactly where wget2 is stuck for you e.g. which line is on the screen when wget2 is stuck, for how long etc.
Thanks. I am running wget from subprocess.Popen Python. I will try to repeat the same event in the terminal with the -d parameter
I tried to catch the stuck several times, but it only succeeded 3 times out of 15:
sometimes site downloaded quickly: about 10-30 second. At some times (when the log files are made) it took about 1.5 minutes and I stopped the work
and i found 1 think: wget2 create // from command with the / at the end for command:
-P /home/user/test/wget_tester/
wget2 create this path:
/home/user/test/wget_tester//obse.silverlock.org
commands in terminal:
wget2 -d --no-cookies --mirror -e robots=off --max-threads=10 -D obse.silverlock.org/ -P /home/user/test/wget_tester/ https://obse.silverlock.org/ -o /home/user/test/wget_tester/obse_2.log
commands in python:
command = 'wget2 -d --no-cookies --mirror -e robots=off --max-threads=10 -D obse.silverlock.org -P /opt/project/worker/download/10 https://obse.silverlock.org/ -o /home/user/test/site_test/wget/logs/obse.log'
command_list = command.split(' ')
proc = Popen(command_list, stdin=PIPE, stdout=PIPE, stderr=PIPE)
proc.wait()
What about this info? i catch this stuck sometimes
I'm on a metered connection. So every download for me takes while. ;)
Last time this command took 5+ minutes for me and you think 1.5 minutes is stuck! So, I can't reproduce this speed/stuck issue at my end. At least not at the moment. But maybe others can?
Just hang in there and maintainers will reply. :)
Since GNU Wget2 1.99.1
we fixed a bunch of bugs. With the latest version from git master this issue is not reproducible.
I hope we soon find some time to make a new release.
Thank! we will wait the release
The new release v2.0.1 is out: https://gitlab.com/gnuwget/wget2/-/tags/v2.0.1
Thanks! We will try
i tried to download http://f4se.silverlock.org from terminal or python(os.system). My wget2 sometimes gets stuck to 5-10 minutes and does not write anything to the log. In the end it loaded succesfully. Sometimes this time can be 10-15 minutes command: wget2 --no-cookies --mirror -e robots=off --max-threads=10 -D f4se.silverlock.org -P /home/user/download/f4se http://f4se.silverlock.org/
Most often this site is downloaded in 2 minutes. most likely there is a timeout wait, but it takes a very long time. Perhaps it should be shortened with --timeout or is it still related to some kind of error? The process just hangs and doesn't log anything.