Open reidsunderland opened 2 weeks ago
try active keyword in credentials.conf ?
active/passive doesn't make a difference in this case :(
I was a little bit wrong with how I was using FTPS.
I was using a URL starting with ftps://
which I realize now isn't supported and I should have been using ftp://example.com tls
.
(I think ftps://
not working is its own bug, the examples in credentials.conf imply that it should work, and we're using it in v2.)
But that still doesn't work. It reveals the actual error though:
2024-06-25 15:38:05,051 [ERROR] 465081 sarracenia.transfer.ftp connect Unable to connect to smos-diss.eo.esa.int (user:ssc.hpc-chp.spc@canada.ca)
2024-06-25 15:38:05,051 [DEBUG] 465081 sarracenia.transfer.ftp connect Exception details:
Traceback (most recent call last):
File "/net/local/home/sunderlandr/sr3/sarracenia/transfer/ftp.py", line 222, in connect
ftp = ftplib.FTP_TLS(self.host,
File "/usr/lib/python3.10/ftplib.py", line 740, in __init__
super().__init__(host, user, passwd, acct,
File "/usr/lib/python3.10/ftplib.py", line 123, in __init__
self.login(user, passwd, acct)
File "/usr/lib/python3.10/ftplib.py", line 745, in login
self.auth()
File "/usr/lib/python3.10/ftplib.py", line 753, in auth
resp = self.voidcmd('AUTH TLS')
File "/usr/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/usr/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/usr/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
ftplib.error_perm: 530 Please login with USER and PASS.
The password doesn't have any special characters that would require encoding, I even tried changing the password to be 100% sure.
Yes, they do still have an FTP server there and that is what we're connecting to on port 21 and getting the 530 error.
If I set port 990 in the URL and do not specify tls
in the config, sr3 will try to connect on port 990 using plain FTP. It just hangs though.
sr3 does not support changing the port for explicit FTPS - in the FTP_TLS section, self.port is not used at all: https://github.com/MetPX/sarracenia/blob/6b24f674282649572734443480bff7fd71d6b051/sarracenia/transfer/ftp.py#L176-L195 (probably a bug we should fix?)
Even trying to specify the port with FTP_TLS outside of sr3, it still hangs/times out:
>>> import ftplib
>>> ftp = ftplib.FTP_TLS()
>>> ftp.encoding = 'utf-8'
>>> ftp.connect('smos-diss.eo.esa.int', port=990, timeout=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.10/ftplib.py", line 162, in connect
self.welcome = self.getresp()
File "/usr/lib/python3.10/ftplib.py", line 244, in getresp
resp = self.getmultiline()
File "/usr/lib/python3.10/ftplib.py", line 230, in getmultiline
line = self.getline()
File "/usr/lib/python3.10/ftplib.py", line 212, in getline
line = self.file.readline(self.maxline + 1)
File "/usr/lib/python3.10/socket.py", line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out
It does work with the modified code in the PR https://github.com/MetPX/sarracenia/pull/1122. That changes the socket used by FTP_TLS to an instance of an ssl.SSLSocket
and then the server allows the connection.
This is implemented in #1122, but has to be configured manually. Peter mentioned on the PR that it would be nice to have the code automatically try to detect and remember which it needs to use (implicit vs explicit) so I'll leave this issue open.
ESA SMOS recently stopped allowing insecure FTP access to their data and now only allows FTPS.
https://smos-diss.eo.esa.int/oads/access/
The existing FTPS code doesn't work and gives this error:
FileZilla can connect:
I tried connecting using ftplib "by hand" and get various problems (
ftplib.error_perm: 530 Please login with USER and PASS.
in most cases).The problem seems to be that ftplib always tries to use port 21. The ESA server does respond, but does not allow us to login (I get the same error with FileZilla when connecting to port 21).
I also tried specifying port 990 like this:
But it just hangs.
This code works great: https://stackoverflow.com/questions/12164470/python-ftp-implicit-tls-connection-issue
So I'll implement this in the FTP transfer class.