search

ArchiveTeam / ftp-queue

Create queue items for ftp-grab.
6 stars 9 forks source link

script frooze for big ammount of time | 550 NONEXISTINGFILE #7

Open raspher opened 6 years ago

raspher commented 6 years ago

lot of jobs complete (incomplete?) with result from log

Wrote HTML-ized index to â.
--2017-12-17 06:18:11--  ftp://ftp.taygeta.com/pub/Linux/kernel/rtlinux/redhat-6.1-rtlinux/RedHat/instimage/usr/X11R6/lib/X11/xkb/symbols/sun/
           => â
Resolving ftp.taygeta.com (ftp.taygeta.com)... 206.55.232.70
Connecting to ftp.taygeta.com (ftp.taygeta.com)|206.55.232.70|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/Linux/kernel/rtlinux/redhat-6.1-rtlinux/RedHat/instimage/usr/X11R6/lib/X11/xkb/symbols/sun ... done.
==> PASV ... done.    ==> LIST ... done.

    [ <=>          ] 301         --.-K/s   in 0s      

2017-12-17 06:18:13 (17.5 MB/s) - â saved [301]

Removed â.
Wrote HTML-ized index to â.

Script stuck here for few hours with 100% CPU usage

550 NONEXISTINGFILEdgdjahxnedadbacxjbc: No such file or directory
550 NONEXISTINGFILEdgdjahxnedadbacxjbc: No such file or directory
15305340894
raspher commented 6 years ago

traceback after killing ( C^c ) this job

python2 ftp_check.py 1 Traceback (most recent call last): File "ftp_check.py", line 48, in if not 'ftp://' + re.search(r'^([^\/]+)', ftp).group(1) + dir + './' in itemslist: KeyboardInterrupt

it can be cos of weak CPU/disk (high wa in top) but god, 2 jobs running with the same task (they stuck) for 5-8 h. (after i wake up...) 1 job still stucks, 2nd finished with 550...

the list were just 100-150MB in size after completion!

i think the script need better information what is going on, after that i can say "it's an issue" or it's not...