Simple server hangs in thread

[oz123]

I am not sure this is a bug, maybe it is. I wrote a plugin for pytest which creates a test fixture with an simple FTPD waiting for requests.

The code is found here: https://github.com/oz123/pytest-localftpserver

My issue is that the call for the server.start() is running while invoking a test module, but it is not while running the test suite. I am not sure how to solve this problem.

Here is an output of test runs:

$ make test 
pytest -s -vvv
====================================================================== test session starts =======================================================================
platform linux -- Python 3.4.3, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /home/oznt/.virtualenvs/bcc2/bin/python3.4
cachedir: .cache
rootdir: /home/oznt/Software/controller_configuration, inifile: pytest.ini
plugins: localftpserver-0.3.1, env-0.6.0
collected 9 items 

tests/test_a_logic.py::test_1_render_mconfig PASSED
tests/test_a_logic.py::test_2_create_bundle PASSED
tests/test_c2_functional.py::test_index PASSED
tests/test_c2_functional.py::test_list PASSED
tests/test_c2_functional.py::test_post PASSED
tests/test_c_ftp_upload.py::test_ftp [I 2016-12-19 23:32:56] >>> starting FTP server on 127.0.0.1:31124, pid=31437 <<<
[I 2016-12-19 23:32:56] concurrency model: async
[I 2016-12-19 23:32:56] masquerade (NAT) address: None
[I 2016-12-19 23:32:56] passive ports: None

This just hangs there indefinitely ...

When I run the last test module by it's own, everything is fun:

 $ pytest -vvv -s tests/test_c_ftp_upload.py 
====================================================================== test session starts =======================================================================
platform linux -- Python 3.4.3, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /home/oznt/.virtualenvs/bcc2/bin/python3.4
cachedir: .cache
rootdir: /home/oznt/Software/controller_configuration, inifile: pytest.ini
plugins: localftpserver-0.3.1, env-0.6.0
collected 2 items 

tests/test_c_ftp_upload.py::test_ftp [I 2016-12-19 23:56:24] >>> starting FTP server on 127.0.0.1:31124, pid=31704 <<<
[I 2016-12-19 23:56:24] concurrency model: async
[I 2016-12-19 23:56:24] masquerade (NAT) address: None
[I 2016-12-19 23:56:24] passive ports: None
[I 2016-12-19 23:56:24] 127.0.0.1:50300-[] FTP session opened (connect)
[I 2016-12-19 23:56:24] 127.0.0.1:50300-[M1] USER 'M1' logged in.
[I 2016-12-19 23:56:24] 127.0.0.1:50300-[M1] CWD /tmp/tmp_j8qsex5 250
[I 2016-12-19 23:56:24] 127.0.0.1:50300-[M1] MKD /tmp/tmp_j8qsex5/FOO 257
[I 2016-12-19 23:56:24] 127.0.0.1:50300-[M1] FTP session closed (disconnect).
PASSED
tests/test_c_ftp_upload.py::test_initiate_logging [I 2016-12-19 23:56:24] 127.0.0.1:50302-[] FTP session opened (connect)
[I 2016-12-19 23:56:24] 127.0.0.1:50302-[M1] USER 'M1' logged in.
[I 2016-12-19 23:56:24] 127.0.0.1:50302-[M1] CWD /tmp/tmp_j8qsex5 250
[I 2016-12-19 23:56:24] 127.0.0.1:50302-[M1] MKD /tmp/tmp_j8qsex5/BCM-LOGFILES 257
PASSED[I 2016-12-19 23:56:24] >>> shutting down FTP server (0 active socket fds) <<<
[I 2016-12-19 23:56:24] 127.0.0.1:50302-[M1] FTP session closed (disconnect).

==================================================================== 2 passed in 0.25 seconds ====================================================================

This is the contents of that test_module:

from ftplib import FTP

from bcc.util import Configurator

import pytest

FTP_USER = "M1"
FTP_PASS = "bachmann"
FTP_PORT = 31124

class DeviceDummy:
    def __init__(self, site, name):
        self.site = site
        self.name = name
        self.user_name = "M1"
        self.password = "bachmann"
        self.net_address = "127.0.0.1"

def test_ftp(ftpserver):
    with FTP() as f:
        f.connect("localhost", port=ftpserver.server_port)
        f.login("M1", "bachmann")
        f.cwd("/")
        f.mkd("FOO")
        f.quit()

def test_initiate_logging(ftpserver):
    dd = DeviceDummy("test-site", "device_foo")
    cfg = Configurator(dd)
    cfg.connect(dd.net_address,
                port=ftpserver.server_port)
    cfg.initiate_logging()

[giampaolo]

Well, I guess it's impossible to figure this out without seeing any code. What does the test FTP server looks like? Is it a thread test FTP server similar to the one used in pyftpdlib test suite?

[oz123]

@giampaolo Thanks for the reply, the code for ftpserver is linked above in my repository. But here is it again: https://raw.githubusercontent.com/oz123/pytest-localftpserver/3a1ddeb85fa4dc379ae2015d1aab5c10601e54a5/pytest_localftpserver/plugin.py

Currently, I changed the code to work with mutliprocessing, and it seems to work better.

[Scripter17]

Having the same issue almost 5 and a half years later

It seems that having abc.py contain the FTPServer object and having xyz.py import it and call threading.Thread(target=abc.server.serve_forever).start() causes the issue. Starting the thread in abc.py doesn't cause an issue

Edit: Actually it seems to throw an error about binding to the same port twice???

  File "C:\Users\...\AppData\Local\Programs\Python\Python310\lib\site-packages\pyftpdlib\servers.py", line 118, in __init__
    self.bind_af_unspecified(address_or_socket)
  File "C:\Users\...\AppData\Local\Programs\Python\Python310\lib\site-packages\pyftpdlib\ioloop.py", line 1018, in bind_af_unspecified
    raise socket.error(err)
OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted

[abdullahshabbir2]

did you find any solution? @Scripter17

[Scripter17]

Sadly no

I'm busy today and tomorrow but I'll see if I can figure it out