test_idle_timeout and test_stou_max_tries sometimes fail

[sbraz]

Hi, This was initially reported here: https://bugs.gentoo.org/758686

=================================== FAILURES ===================================
______________________ ThreadedFTPTests.test_idle_timeout ______________________

self = <pyftpdlib.test.test_functional.ThreadedFTPTests testMethod=test_idle_timeout>

    def test_idle_timeout(self):
        # Test control channel timeout.  The client which does not send
        # any command within the time specified in FTPHandler.timeout is
        # supposed to be kicked off.
        with self.server.lock:
            self.server.handler.timeout = 0.1

        try:
>           self.client.quit()

pyftpdlib/test/test_functional.py:2606: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.6/ftplib.py:665: in quit
    resp = self.voidcmd('QUIT')
/usr/lib64/python3.6/ftplib.py:278: in voidcmd
    return self.voidresp()
/usr/lib64/python3.6/ftplib.py:251: in voidresp
    resp = self.getresp()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ftplib.FTP object at 0x7f7e4a2ad630>

    def getresp(self):
        resp = self.getmultiline()
        if self.debugging:
            print('*resp*', self.sanitize(resp))
        self.lastresp = resp[:3]
        c = resp[:1]
        if c in {'1', '2', '3'}:
            return resp
        if c == '4':
>           raise error_temp(resp)
E           ftplib.error_temp: 421 Control connection timed out.

/usr/lib64/python3.6/ftplib.py:244: error_temp
_____________________ ThreadedFTPTests.test_stou_max_tries _____________________

self = <pyftpdlib.test.test_functional.ThreadedFTPTests testMethod=test_stou_max_tries>

    def test_stou_max_tries(self):
        # Emulates case where the max number of tries to find out a
        # unique file name when processing STOU command gets hit.

        class TestFS(AbstractedFS):

            def mkstemp(self, *args, **kwargs):
                raise IOError(errno.EEXIST,
                              "No usable temporary file name found")

        with self.server.lock:
            self.server.handler.abstracted_fs = TestFS
        try:
>           self.client.quit()

pyftpdlib/test/test_functional.py:2590: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.6/ftplib.py:665: in quit
    resp = self.voidcmd('QUIT')
/usr/lib64/python3.6/ftplib.py:278: in voidcmd
    return self.voidresp()
/usr/lib64/python3.6/ftplib.py:251: in voidresp
    resp = self.getresp()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ftplib.FTP object at 0x7f7e497b7b38>

    def getresp(self):
        resp = self.getmultiline()
        if self.debugging:
            print('*resp*', self.sanitize(resp))
        self.lastresp = resp[:3]
        c = resp[:1]
        if c in {'1', '2', '3'}:
            return resp
        if c == '4':
>           raise error_temp(resp)
E           ftplib.error_temp: 421 Control connection timed out.

/usr/lib64/python3.6/ftplib.py:244: error_temp
=========================== short test summary info ============================
FAILED pyftpdlib/test/test_functional.py::ThreadedFTPTests::test_idle_timeout
FAILED pyftpdlib/test/test_functional.py::ThreadedFTPTests::test_stou_max_tries
==== 2 failed, 704 passed, 55 skipped, 16 deselected in 3326.70s (0:55:26) =====

Should we mark both these tests as unstable?

[giampaolo]

Hello. Yeah, functional tests are not in a good shape and should be rewritten. The main problem appears to be the thread-based test server, which appears to be the cause of these intermittent failures. I should try to use a multi-process based test server instead.

[giampaolo]

Should we mark both these tests as unstable?

I would say yes, in detail ThreadedFTPTests class in ŧest/test_functional.py.

[giampaolo]

544 should have fixed this specific issue (via retry on failure), but I still do not consider all tests as 100% reliable.