550 Failed to change directory error on Linux

[lincoln-harris]

I'm getting a strange error message when I try to use ppx.download(). Here is the full traceback:

--------------------------------------------------------------------
error_temp                         Traceback (most recent call last)
<ipython-input-8-7a03192dc20b> in <module>
      2 proj = ppx.find_project("PXD011961", local="data/raw")
      3 #proj.remote_files()
----> 4 proj.download("proteinGroups.txt")

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/project.py in download(self, files, force_, silent)
    232         """
    233         files = utils.listify(files)
--> 234         in_remote = [f in self.remote_files() for f in files]
    235         if not all(in_remote):
    236             missing = [f for i, f in zip(in_remote, files) if not i]

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/project.py in <listcomp>(.0)
    232         """
    233         files = utils.listify(files)
--> 234         in_remote = [f in self.remote_files() for f in files]
    235         if not all(in_remote):
    236             missing = [f for i, f in zip(in_remote, files) if not i]

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/project.py in remote_files(self, glob)
    159         """
    160         if self.fetch or self._remote_files is None:
--> 161             self._remote_files = self._parser.files
    162 
    163         if glob is not None:

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/ftp.py in files(self)
    242         """List the files form the FTP connection"""
    243         if self._files is None:
--> 244             self._get_files()
    245             self._files.sort()
    246 

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/ftp.py in _get_files(self)
    183     def _get_files(self):
    184         """Recursively list files from the FTP connection."""
--> 185         self.connect()
    186         self._files, self._dirs = self._with_reconnects(self._parse_files)
    187         self._depth = 0

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/ftp.py in connect(self)
     81     def connect(self):
     82         """Connect to the FTP server, with reconnects on failure."""
---> 83         self._with_reconnects(self._connect)
     84 
     85     def quit(self):

~/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/ftp.py in _with_reconnects(self, func, *args, **kwargs)
    111 
    112         raise error_temp(
--> 113             f"Failed after {self.max_reconnects} reconnect(s), "
    114             f"the last error was: {last_err}"
    115         )

error_temp: Failed after 10 reconnect(s), the last error was: 550 Failed to change directory.

For some reason I'm only seeing this on my linux system. MacOS is working fine.

I'm seeing this when I use both the python API and the CLI. Here is an example command:

$ ppx --local data {PRIDE_proj} proteinGroups.txt

[wfondrie]

Can you provide the full command for a project where this occurs? The error appears to be from within an IPython sessions, although you said that you're using the CLI:

--------------------------------------------------------------------
error_temp                         Traceback (most recent call last)
<ipython-input-8-7a03192dc20b> in <module>
      2 proj = ppx.find_project("PXD011961", local="data/raw")
      3 #proj.remote_files()
----> 4 proj.download("proteinGroups.txt")

Also, can you share what version is installed? You can get this either with conda list ppx or pip show ppx depending on how you installed it.

[lincoln-harris]

Thats right, the above error message came while using an ipython session. Here is the full traceback when I run the following CLI command on Linux:

ppx --local data PXD019445 proteinGroups.txt

The ppx version is 1.2.3.

Traceback (most recent call last):
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connection.py", line 182, in _new_conn
    % (self.host, self.timeout),
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7fc719014750>, 'Connection to proteomecentral.proteomexchange.org timed out. (connect timeout=10.0)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='proteomecentral.proteomexchange.org', port=80): Max retries exceeded with url: /cgi/GetDataset?ID=PXD019445&outputMode=JSON&test=no (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fc719014750>, 'Connection to proteomecentral.proteomexchange.org timed out. (connect timeout=10.0)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/bin/ppx", line 10, in <module>
    sys.exit(main())
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/ppx.py", line 80, in main
    proj = find_project(args.identifier, args.local)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/factory.py", line 178, in find_project
    return PXDFactory(identifier, **kwargs).find()
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/ppx/factory.py", line 56, in __init__
    res = requests.get(self.rest, params=params, timeout=self._timeout)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute/lib/python3.7/site-packages/requests/adapters.py", line 504, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='proteomecentral.proteomexchange.org', port=80): Max retries exceeded with url: /cgi/GetDataset?ID=PXD019445&outputMode=JSON&test=no (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fc719014750>, 'Connection to proteomecentral.proteomexchange.org timed out. (connect timeout=10.0)'))

[wfondrie]

Thanks @lincoln-harris. This actually appears to be a different error than the original one you posted.

Here it looks like the connection from your Linux machine is timing out. A few releases ago I added a timeout parameter to the Python API (see #13), but I forgot to do so with the CLI. I'll add it and raising it should fix this latest error.

[wfondrie]

ppx v1.2.4 is now up on PyPI (and soon bioconda) 🎉

After updating, you can increase the timeout duration from the CLI, which should resolve the latter problem:

ppx --local data --timeout 10 PXD019445 proteinGroups.txt

As for the former error, PRIDE appears to be changing their FTP URLs yet again (see #18, which I thought was resolved). Unfortunately there's not much I can do until they work this out.

[lincoln-harris]

Oh I see. After installing version 1.2.4 the second error goes away, but I start to see the first one again.

$ ppx --local data --timeout 10 PXD019445 proteinGroups.txt :

Traceback (most recent call last):
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/bin/ppx", line 8, in <module>
    sys.exit(main())
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/lib/python3.8/site-packages/ppx/ppx.py", line 93, in main
    remote_files = proj.remote_files()
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/lib/python3.8/site-packages/ppx/project.py", line 161, in remote_files
    self._remote_files = self._parser.files
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/lib/python3.8/site-packages/ppx/ftp.py", line 244, in files
    self._get_files()
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/lib/python3.8/site-packages/ppx/ftp.py", line 185, in _get_files
    self.connect()
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/lib/python3.8/site-packages/ppx/ftp.py", line 83, in connect
    self._with_reconnects(self._connect)
  File "/net/noble/vol7/home/lincolnh/miniconda3/envs/ms-impute-r/lib/python3.8/site-packages/ppx/ftp.py", line 112, in _with_reconnects
    raise error_temp(
ftplib.error_temp: Failed after 10 reconnect(s), the last error was: 550 Failed to change directory.

[wfondrie]

Yes, this is a problem on PRIDE's end currently. Hopefully they'll fix it soon.

[wfondrie]

I think this should be resolved. Please reopen the issue if you're still having problems with the latest version of ppx!