lavv17
commented
4 years ago It seems that mput tries to transfer each file before exiting. It needs to stop early.
— Alexander.
On 27 May 2020, at 00:53, Chris Severance notifications@github.com wrote:
I have an ftps-explicit server that logs in fine but hangs when I type ls or any other file transfer command. My shell script waits a long time on this lftp command.
lftp 'ftp://[redacted]:800' -e 'set net:limit-rate 100K; set net:timeout 10; set xfer:timeout 30; set net:max-retries 1; set cmd:fail-exit true; set cmd:interactive false; set log:file/xfer "./xfer.sh.log"; set log:enabled/xfer true; set ftp:ssl-protect-data true; set ftp:ssl-force true; set ssl:verify-certificate off; set ssl:ca-file "/etc/ssl/certs/ca-certificates.crt"; set net:timeout 30; mput -E -O "" "*"; set log:enabled/xfer false; ls ""; bye' -u '[redacted],xxxxxxxx' lftp shows this for every file to be sent: [Making data connection..] [Delaying before reconnect: ...]. I didn't notice the problem when the delay was short and there were only a few files waiting. Now that there are 99 files waiting the delay is so long that the run one hour later detects the previous run hasn't completed.
Turns out this is related to a very old bug in cmd:fail-exit I noticed in an old compile for SCO lftp-3.3.0.tar.bz2. cmd:fail-exit does not apply to mput unless some other data transfer command like ls comes first. Adding ls before mput fixes the problem. Looks like the bug still exists.
$ pacman -Q lftp lftp 4.9.1-1 $ lftp -v LFTP | Version 4.9.1 | Copyright (c) 1996-2020 Alexander V. Lukyanov ... Libraries used: Expat 2.2.9, GnuTLS 3.6.13, idn2 2.3.0, Readline 8.0, zlib 1.2.11 Debug with ls command added before mput:
---- Resolving host address... ---- 1 address found: [redacted] ---- Connecting to [redacted] ([redacted]) port 800 <--- 220-FileZilla Server 0.9.56 beta <--- 220 Welcome to [redacted] ---> FEAT <--- 211-Features: <--- MDTM <--- REST STREAM <--- SIZE <--- MLST type;size;modify*; <--- MLSD <--- AUTH SSL <--- AUTH TLS <--- PROT <--- PBSZ <--- UTF8 <--- CLNT <--- MFMT <--- EPSV <--- EPRT <--- 211 End ---> AUTH TLS <--- 234 Using authentication type TLS ---> CLNT lftp/4.9.1 Certificate: [redacted] Issued by: [redacted] WARNING: Certificate verification: Not trusted ([redacted]) WARNING: Certificate verification: Expired ([redacted]) WARNING: Certificate verification: certificate common name doesn't match requested host name ‘[redacted]’ ([redacted]) <--- 200 Don't care ---> OPTS UTF8 ON <--- 202 UTF8 mode is always enabled. No need to send this command. ---> USER [redacted] <--- 331 Password required for [redacted] ---> PASS [redacted] <--- 230 Logged on ---> PWD <--- 257 "/" is current directory. ---> PBSZ 0 <--- 200 PBSZ=0 ---> PROT P <--- 200 Protection level set to P ---> PASV <--- 227 Entering Passive Mode ([redacted],217,213) ---- Connecting data socket to ([redacted]) port 55765 ls: no progress timeout ---> LIST ---> ABOR ---- Closing aborted data socket ---- Closing control socket Debug without ls command, ^C during the second file:
---- Resolving host address... ---- 1 address found: [redacted] ---- Connecting to [redacted] ([redacted]) port 800 <--- 220-FileZilla Server 0.9.56 beta <--- 220 Welcome to [redacted] ---> FEAT <--- 211-Features: <--- MDTM <--- REST STREAM <--- SIZE <--- MLST type;size;modify*; <--- MLSD <--- AUTH SSL <--- AUTH TLS <--- PROT <--- PBSZ <--- UTF8 <--- CLNT <--- MFMT <--- EPSV <--- EPRT <--- 211 End ---> AUTH TLS <--- 234 Using authentication type TLS ---> CLNT lftp/4.9.1 Certificate: [redacted] Issued by: [redacted] WARNING: Certificate verification: Not trusted ([redacted]) WARNING: Certificate verification: Expired ([redacted]) WARNING: Certificate verification: certificate common name doesn't match requested host name ‘[redacted]’ ([redacted]) <--- 200 Don't care ---> OPTS UTF8 ON <--- 202 UTF8 mode is always enabled. No need to send this command. ---> USER [redacted] <--- 331 Password required for [redacted] ---> PASS [redacted] <--- 230 Logged on ---> PWD <--- 257 "/" is current directory. ---> PBSZ 0 <--- 200 PBSZ=0 ---> PROT P <--- 200 Protection level set to P ---> TYPE I <--- 200 Type set to I ---> PASV <--- 227 Entering Passive Mode ([redacted],217,233) ---- Connecting data socket to ([redacted]) port 55785 *** Timeout - reconnecting ---- Closing data socket ---- Closing control socket mput: [redacted]: Fatal error: max-retries exceeded ---- Connecting to [redacted] ([redacted]) port 800 <--- 220-FileZilla Server 0.9.56 beta <--- 220 Welcome to [redacted] ---> FEAT <--- 211-Features: <--- MDTM <--- REST STREAM <--- SIZE <--- MLST type;size;modify; <--- MLSD <--- AUTH SSL <--- AUTH TLS <--- PROT <--- PBSZ <--- UTF8 <--- CLNT <--- MFMT <--- EPSV <--- EPRT <--- 211 End ---> AUTH TLS <--- 234 Using authentication type TLS ---> CLNT lftp/4.9.1 Certificate: [redacted] Issued by: [redacted] WARNING: Certificate verification: Not trusted ([redacted]) WARNING: Certificate verification: Expired ([redacted]) WARNING: Certificate verification: certificate common name doesn't match requested host name ‘[redacted]’ ([redacted]) <--- 200 Don't care ---> OPTS UTF8 ON <--- 202 UTF8 mode is always enabled. No need to send this command. ---> USER [redacted] <--- 331 Password required for [redacted] ---> PASS [redacted] <--- 230 Logged on ---> PBSZ 0 <--- 200 PBSZ=0 ---> PROT P <--- 200 Protection level set to P ---> TYPE I <--- 200 Type set to I ---> PASV <--- 227 Entering Passive Mode ([redacted],214,223) ---- Connecting data socket to ([redacted]) port 55007 ^C The hanging problem from an IP address mismatch between Connecting to [redacted] and Entering Passive Mode ([redacted],. This is not related to lftp.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.
I have an ftps-explicit server that logs in fine but hangs when I type
lsor any other file transfer command. My shell script waits a long time on this lftp command.lftp shows this for every file to be sent:
[Making data connection..] [Delaying before reconnect: ...]. I didn't notice the problem when the delay was short and there were only a few files waiting. Now that there are 99 files waiting the delay is so long that the run one hour later detects the previous run hasn't completed.Turns out this is related to a very old bug in
cmd:fail-exitI noticed in an old compile for SCO lftp-3.3.0.tar.bz2.cmd:fail-exitdoes not apply tomputunless some other data transfer command likelscomes first. Addinglsbeforemputfixes the problem. Looks like the bug still exists.Debug with
lscommand added beforemput:Debug without
lscommand, ^C during the second file:The hanging problem from an IP address mismatch between
Connecting to [redacted]andEntering Passive Mode ([redacted],. This is not related tolftp.Interestingly enough the advice on the SCO page lead me to a successful connect. Enabling
ftp:ignore-pasv-addressgets the files through.Linut's SCO stuff
Source distribution of lftp with compiled binaries included for SCO Open Server. Built on Open Server 5.0.6 with gnutools & gwxlibs. Should run on any version that can install oss646 & gwxlibs
Although lftp has lots of special features that I'm sure are good, even life savers, in lots of special situations, the particular thing that makes it almost unique and which solves a real humdinger problem for me, is it includes an option to let you specify an arbitrary ip address for it to use in PORT commands.
The only other client I've found that allows this is curl, but curl is otherwise very limited and not very handy for ftp work. For one thing, curl can't download "*" and produce seperate filenames locally like a proper ftp client.
I have a situation where the client unix box is behind a common nat router that does the normal magic ftp packet editing, but like most routers it only does ths on the standard ftp tcp ports! This client however needs to ftp in to a server that is running on a non-standard tcp port, and, the server does not allow PASV, only PORT. Geniuses! But they are a customer of my customer, and so I can't very well tell their edi people to go read "Ftp for Dummies".
So, the only way to ftp to them is one of the following: a) Replace the client side router with an expensive and hard to administer Cisco. Gross Overkill! b) Remove the router and put the unix box right on the public net. Just Plain Gross! c) Know the public ip of the client side router and use an ftp client that allows you to specify that ip for it to use in PORT commands.
sigh "Some days it's just not worth chewing through the restraints..." :)