giampaolo / pyftpdlib

Extremely fast and scalable Python FTP server library
MIT License
1.67k stars 266 forks source link

Multiprocess FTP server reached max_cons and throw an exception #506

Open MrZXR opened 5 years ago

MrZXR commented 5 years ago

This is a service that started a long time ago, works well until today. I use the 1.5.4 version of Multiprocess FTP server with python2.6, and when the total number of connections reaches max_cons, the new connection causes the program to throw an exception like this:

    Traceback (most recent call last):
      File "/usr/lib64/python2.6/asyncore.py", line 78, in read
        obj.handle_read_event()
      File "/usr/lib64/python2.6/asyncore.py", line 428, in handle_read_event
        self.handle_read()
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/ioloop.py", line 920, in handle_read
        asynchat.async_chat.handle_read(self)
      File "/usr/lib64/python2.6/asynchat.py", line 158, in handle_read
        self.found_terminator()
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/handlers.py", line 1437, in found_terminator
        self.pre_process_command(line, cmd, arg)
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/handlers.py", line 1554, in pre_process_command
        self.process_command(cmd, arg, **kwargs)
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/handlers.py", line 1565, in process_command
        method(*args, **kwargs)
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/handlers.py", line 2074, in ftp_PASV
        self._make_epasv(extmode=False)
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/handlers.py", line 1986, in _make_epasv
        if not self.server._accept_new_cons():
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/servers.py", line 136, in _accept_new_cons
        return self._map_len() <= self.max_cons
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/servers.py", line 326, in _map_len
        self._refresh_tasks()
      File "/root/git/ftp/venv/lib/python2.6/site-packages/pyftpdlib/servers.py", line 339, in _refresh_tasks
        if not t.is_alive():
      File "/usr/lib64/python2.6/multiprocessing/process.py", line 133, in is_alive
        assert self._parent_pid == os.getpid(), 'can only test a child process'
    AssertionError: can only test a child process

When this part of the servers.py code is annotated, there is no problem

    def _map_len(self):
        #if len(self._active_tasks) >= self.max_cons:
        #    # Since refresh()ing is a potentially expensive operation
        #    # (O(N)) do it only if we're exceeding max connections
        #    # limit. Other than in here, tasks are refreshed every 10
        #    # seconds anyway.
        #    self._refresh_tasks()
        return len(self._active_tasks)

I think the refresh operation here is the problem. Periodic refresh works well, only the first new connection when the threshold is reached will raise exec