ERROR - MLSD returned unsupported type: None

[bDrwx]

I'am running synchronization task by sceduler. Somtime this task start crushing with this error. I fix this problem by delete meta file on remote ftp server. What is the root cause?

[2020-12-03 14:18:26,153] {{taskinstance.py:1128}} ERROR - MLSD returned unsupported type: None
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 966, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 113, in execute
    return_value = self.execute_callable()
  File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 118, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/usr/local/airflow/dags/get_cdr_SKPTUS.py", line 111, in export_to_asr
    s.run()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/synchronizers.py", line 1106, in run
    res = super(UploadSynchronizer, self).run()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/synchronizers.py", line 827, in run
    res = super(BiDirSynchronizer, self).run()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/synchronizers.py", line 213, in run
    res = self._sync_dir()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/synchronizers.py", line 485, in _sync_dir
    remote_entries = self.remote.get_dir()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/ftp_target.py", line 514, in get_dir
    self._ftp_retrlines_native("MLSD", _addline, encoding)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/ftp_target.py", line 772, in _ftp_retrlines_native
    self.ftp.retrbinary(command, _on_read_chunk)
  File "/usr/local/lib/python3.7/ftplib.py", line 447, in retrbinary
    callback(data)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/ftp_target.py", line 766, in _on_read_chunk
    _on_read_line(item)  # + LF)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/ftp_target.py", line 752, in _on_read_line
    callback(status, line_decoded)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/ftpsync/ftp_target.py", line 505, in _addline
    "MLSD returned unsupported type: {!r}".format(res_type)
NotImplementedError: MLSD returned unsupported type: None

[antonzhelyazkov]

Hi @bDrwx,

I got the same problem with version 3. It seems that this problem is solved in version 4. Could you try with version 4?

I hope to see stable version 4 sooner.

[bDrwx]

Thank you. I will try to update and report by result.

[bDrwx]

I found that my problem repeats the error described in #42 But i use pyftpsync directly from the source code:

def export_to_asr():
    """ Export data to remote FTP """
    local = FsTarget("data/path/out")
    remote = FtpTarget("/", FTP_IP, username=FTP_USER, password=FTP_PASS, tls=False)
    opts = {"resolve": "skip", "verbose": 1}
    s = UploadSynchronizer(local, remote, opts)
    s.run()

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[AnyOfYou]

In my case, sometimes the "chunk" in ftp_target.py's _on_read_chunk() will start with "\n", which make the 'item' in _on_read_line() become a empty string, then cause the "MLSD returned unsupported type: None" error. https://imgur.com/biQ9B4V

I can fix that with adding "chunk = chunk.lstrip()" line after "chunk = buffer + chunk" line in _on_read_chunk() method, probably have better way.

pyftpsync version 3.1.0, with pyftpdlib version 1.5.6 as ftp server.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.