search

fsspec / filesystem_spec

A specification that python filesystems should adhere to.
BSD 3-Clause "New" or "Revised" License
1.05k stars 362 forks source link

2024.10.0: pytest fails in fsspec/implementations/tests/test_ftp.py units #1730

Open kloczek opened 1 month ago

kloczek commented 1 month ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.10.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.10.0-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 rootdir: /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0 configfile: pyproject.toml plugins: asyncio-0.23.8, mock-3.14.0, vcr-1.0.2 asyncio: mode=strict collected 1737 items / 9 skipped fsspec/implementations/tests/local/local_test.py ....................... [ 1%] ........................................................................ [ 5%] ........................................ [ 7%] fsspec/implementations/tests/memory/memory_test.py ..................... [ 8%] ........................................................................ [ 13%] .......................................... [ 15%] fsspec/implementations/tests/test_archive.py ........................... [ 17%] ..........................................................ssssssssssssss [ 21%] sss [ 21%] fsspec/implementations/tests/test_cached.py ............................ [ 23%] ..............ssssssssssss................................ [ 26%] fsspec/implementations/tests/test_common.py ssss [ 26%] fsspec/implementations/tests/test_data.py .. [ 26%] fsspec/implementations/tests/test_dirfs.py ............................. [ 28%] ........................................................................ [ 32%] ................................. [ 34%] fsspec/implementations/tests/test_ftp.py FFFFFFFF........... [ 35%] fsspec/implementations/tests/test_http.py .............................. [ 37%] .......................... [ 38%] fsspec/implementations/tests/test_libarchive.py s [ 38%] fsspec/implementations/tests/test_local.py .s........................... [ 40%] .s............................................................ss........ [ 44%] ss.sssss.......sss....s.......................... [ 47%] fsspec/implementations/tests/test_memory.py ............................ [ 49%] .. [ 49%] fsspec/implementations/tests/test_reference.py .................s.....ss [ 50%] ..sss [ 50%] fsspec/implementations/tests/test_sftp.py ssssssssssss [ 51%] fsspec/implementations/tests/test_tar.py ......................... [ 53%] fsspec/implementations/tests/test_webhdfs.py sssssssssss [ 53%] fsspec/implementations/tests/test_zip.py ................. [ 54%] fsspec/tests/test_api.py ...............x...... [ 55%] fsspec/tests/test_async.py .........s... [ 56%] fsspec/tests/test_caches.py ............................................ [ 59%] ........................................................................ [ 63%] ............................. [ 64%] fsspec/tests/test_callbacks.py ........ [ 65%] fsspec/tests/test_compression.py .....s [ 65%] fsspec/tests/test_config.py ....... [ 66%] fsspec/tests/test_core.py .............................................. [ 68%] ...s.......s [ 69%] fsspec/tests/test_file.py ........... [ 70%] fsspec/tests/test_generic.py ...... [ 70%] fsspec/tests/test_mapping.py ................. [ 71%] fsspec/tests/test_parquet.py sssssssssssssssssssssssssssssssssssssssssss [ 73%] ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 78%] ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 82%] sssss [ 82%] fsspec/tests/test_registry.py ......s [ 82%] fsspec/tests/test_spec.py ....................x......................... [ 85%] ........................................................................ [ 89%] ........................................................................ [ 93%] ....................................... [ 96%] fsspec/tests/test_utils.py ............................................. [ 98%] ...................... [100%] =================================== FAILURES =================================== _____________________________ test_tls[False-FTP] ______________________________ ftp = ('localhost', 2121), tls = False, exp_cls = @pytest.mark.parametrize( "tls,exp_cls", ( (False, FTP), (True, FTP_TLS), ), ) def test_tls(ftp, tls, exp_cls): host, port = ftp > fs = FTPFileSystem(host, port, tls=tls) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError ____________________________ test_tls[True-FTP_TLS] ____________________________ ftp = ('localhost', 2121), tls = True, exp_cls = @pytest.mark.parametrize( "tls,exp_cls", ( (False, FTP), (True, FTP_TLS), ), ) def test_tls(ftp, tls, exp_cls): host, port = ftp > fs = FTPFileSystem(host, port, tls=tls) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError _____________________________ test_basic[False--] ______________________________ ftp = ('localhost', 2121), tls = False, username = '', password = '' @pytest.mark.parametrize( "tls,username,password", ( (False, "", ""), (True, "", ""), (False, "user", "pass"), (True, "user", "pass"), ), ) def test_basic(ftp, tls, username, password): host, port = ftp > fs = FTPFileSystem(host, port, username, password, tls=tls) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError ______________________________ test_basic[True--] ______________________________ ftp = ('localhost', 2121), tls = True, username = '', password = '' @pytest.mark.parametrize( "tls,username,password", ( (False, "", ""), (True, "", ""), (False, "user", "pass"), (True, "user", "pass"), ), ) def test_basic(ftp, tls, username, password): host, port = ftp > fs = FTPFileSystem(host, port, username, password, tls=tls) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError _________________________ test_basic[False-user-pass] __________________________ ftp = ('localhost', 2121), tls = False, username = 'user', password = 'pass' @pytest.mark.parametrize( "tls,username,password", ( (False, "", ""), (True, "", ""), (False, "user", "pass"), (True, "user", "pass"), ), ) def test_basic(ftp, tls, username, password): host, port = ftp > fs = FTPFileSystem(host, port, username, password, tls=tls) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError __________________________ test_basic[True-user-pass] __________________________ ftp = ('localhost', 2121), tls = True, username = 'user', password = 'pass' @pytest.mark.parametrize( "tls,username,password", ( (False, "", ""), (True, "", ""), (False, "user", "pass"), (True, "user", "pass"), ), ) def test_basic(ftp, tls, username, password): host, port = ftp > fs = FTPFileSystem(host, port, username, password, tls=tls) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError _______________________________ test_not_cached ________________________________ ftp = ('localhost', 2121) def test_not_cached(ftp): host, port = ftp > fs = FTPFileSystem(host, port) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:65: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError ____________________________ test_ls_root_dircache _____________________________ ftp = ('localhost', 2121) def test_ls_root_dircache(ftp): host, port = ftp > fs = FTPFileSystem(host, port) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:72: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__ obj = super().__call__(*args, **kwargs) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__ self._connect() /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect self.ftp.connect(self.host, self.port) /usr/lib64/python3.10/ftplib.py:158: in connect self.sock = socket.create_connection((self.host, self.port), self.timeout, /usr/lib64/python3.10/socket.py:845: in create_connection raise err _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = ('localhost', 2121), timeout = 30, source_address = None def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None): """Connect to *address* and return the socket object. Convenience function. Connect to *address* (a 2-tuple ``(host, port)``) and return the socket object. Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by :func:`getdefaulttimeout` is used. If *source_address* is set it must be a tuple of (host, port) for the socket to bind as a source address before making the connection. A host of '' or port 0 tells the OS to use the default. """ host, port = address err = None for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) if source_address: sock.bind(source_address) > sock.connect(sa) E ConnectionRefusedError: [Errno 111] Connection refused /usr/lib64/python3.10/socket.py:833: ConnectionRefusedError ================================== XFAILURES =================================== __________________________ test_multilevel_chained_fs __________________________ @pytest.mark.xfail(reason="see issue #334", strict=True) def test_multilevel_chained_fs(): """This test reproduces fsspec/filesystem_spec#334""" import zipfile d1 = tempfile.mkdtemp() f1 = os.path.join(d1, "f1.zip") with zipfile.ZipFile(f1, mode="w") as z: # filename, content z.writestr("foo.txt", "foo.txt") z.writestr("bar.txt", "bar.txt") # We expected this to be the correct syntax > with pytest.raises(IsADirectoryError): E Failed: DID NOT RAISE fsspec/tests/test_api.py:252: Failed __________________________________ test_find ___________________________________ @pytest.mark.xfail def test_find(): """Test .find() method on debian server (ftp, https) with constant folder""" filesystem, host, test_path = ( FTPFileSystem, "ftp.fau.de", "ftp://ftp.fau.de/debian-cd/current/amd64/log/success", ) test_fs = filesystem(host) filenames_ftp = test_fs.find(test_path) > assert filenames_ftp E assert [] fsspec/tests/test_spec.py:699: AssertionError =============================== warnings summary =============================== fsspec/tests/test_async.py::test_run_coros_in_chunks /usr/lib64/python3.10/asyncio/tasks.py:596: RuntimeWarning: coroutine 'test_run_coros_in_chunks..runner' was never awaited await __sleep0() Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. fsspec/tests/test_async.py::test_async_streamed_file_write /usr/lib64/python3.10/functools.py:58: RuntimeWarning: coroutine 'test_run_coros_in_chunks..runner' was never awaited getattr(wrapper, attr).update(getattr(wrapped, attr, {})) Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ SKIPPED [1] fsspec/implementations/tests/test_arrow.py:5: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_dask.py:5: could not import 'distributed': No module named 'distributed' SKIPPED [1] fsspec/implementations/tests/test_dbfs.py:35: These tests need to be re-recorded. SKIPPED [1] fsspec/implementations/tests/test_git.py:11: could not import 'pygit2': No module named 'pygit2' SKIPPED [1] fsspec/implementations/tests/test_jupyter.py:10: could not import 'notebook': No module named 'notebook' SKIPPED [1] fsspec/implementations/tests/test_smb.py:15: could not import 'smbprotocol': No module named 'smbprotocol' SKIPPED [1] fsspec/tests/test_downstream.py:3: could not import 's3fs': No module named 's3fs' SKIPPED [1] fsspec/tests/test_fuse.py:9: could not import 'fuse': No module named 'fuse' SKIPPED [1] fsspec/tests/test_gui.py:3: could not import 'panel': No module named 'panel' SKIPPED [18] fsspec/implementations/tests/test_archive.py:45: could not import 'libarchive': No module named 'libarchive' SKIPPED [6] fsspec/implementations/tests/test_cached.py:878: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [6] fsspec/implementations/tests/test_cached.py:892: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_common.py:10: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [3] fsspec/implementations/tests/test_common.py:21: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_local.py:135: could not import 's3fs': No module named 's3fs' SKIPPED [1] fsspec/implementations/tests/test_local.py:287: could not import 'cloudpickle': No module named 'cloudpickle' SKIPPED [2] fsspec/implementations/tests/test_local.py:785: Windows only SKIPPED [2] fsspec/implementations/tests/test_local.py:801: Windows only SKIPPED [5] fsspec/implementations/tests/test_local.py:823: Windows only SKIPPED [3] fsspec/implementations/tests/test_local.py:880: Windows only SKIPPED [1] fsspec/implementations/tests/test_local.py:895: Windows only SKIPPED [1] fsspec/implementations/tests/test_reference.py:433: could not import 'zarr': No module named 'zarr' SKIPPED [1] fsspec/implementations/tests/test_reference.py:579: could not import 'fastparquet': No module named 'fastparquet' SKIPPED [1] fsspec/implementations/tests/test_reference.py:625: could not import 'fastparquet': No module named 'fastparquet' SKIPPED [1] fsspec/implementations/tests/test_reference.py:733: could not import 'zarr': No module named 'zarr' SKIPPED [2] fsspec/implementations/tests/test_reference.py:766: could not import 'kerchunk': No module named 'kerchunk' SKIPPED [1] fsspec/implementations/tests/test_sftp.py:71: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:86: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:104: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:143: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:152: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:183: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:205: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:222: docker run not available SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:43: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:49: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:57: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:67: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:84: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:100: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:121: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:140: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:153: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:163: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:177: htcluster not found SKIPPED [1] fsspec/tests/test_async.py:134: only for windows SKIPPED [1] fsspec/tests/test_compression.py:134: could not import 'snappy': No module named 'snappy' SKIPPED [1] fsspec/tests/test_core.py:357: could not import 's3fs': No module named 's3fs' SKIPPED [1] fsspec/tests/test_core.py:471: could not import 'adlfs': No module named 'adlfs' SKIPPED [64] fsspec/tests/test_parquet.py:37: fastparquet not found SKIPPED [64] fsspec/tests/test_parquet.py:37: pyarrow not found SKIPPED [64] fsspec/tests/test_parquet.py:37: No parquet engine (fastparquet or pyarrow) found SKIPPED [1] fsspec/tests/test_registry.py:129: could not import 's3fs': No module named 's3fs' XFAIL fsspec/tests/test_api.py::test_multilevel_chained_fs - see issue #334 XFAIL fsspec/tests/test_spec.py::test_find FAILED fsspec/implementations/tests/test_ftp.py::test_tls[False-FTP] - Connec... FAILED fsspec/implementations/tests/test_ftp.py::test_tls[True-FTP_TLS] - Con... FAILED fsspec/implementations/tests/test_ftp.py::test_basic[False--] - Connec... FAILED fsspec/implementations/tests/test_ftp.py::test_basic[True--] - Connect... FAILED fsspec/implementations/tests/test_ftp.py::test_basic[False-user-pass] FAILED fsspec/implementations/tests/test_ftp.py::test_basic[True-user-pass] FAILED fsspec/implementations/tests/test_ftp.py::test_not_cached - Connection... FAILED fsspec/implementations/tests/test_ftp.py::test_ls_root_dircache - Conn... = 8 failed, 1452 passed, 284 skipped, 2 xfailed, 2 warnings in 103.85s (0:01:43) = /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-970f0ce1-2bfe-4e93-9c9c-e0210285fe29/test_discovery_via_path_in_non0 : [Errno 39] Directory not empty: 'test_discovery_via_path_in_non0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-970f0ce1-2bfe-4e93-9c9c-e0210285fe29 : [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-970f0ce1-2bfe-4e93-9c9c-e0210285fe29' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-eec6b57a-4bc2-40fa-8c57-7bef6d50de7c/test_discovery_via_path_in_non0 : [Errno 39] Directory not empty: 'test_discovery_via_path_in_non0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-eec6b57a-4bc2-40fa-8c57-7bef6d50de7c : [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-eec6b57a-4bc2-40fa-8c57-7bef6d50de7c' warnings.warn( Unclosed client session client_session: ```
List of installed modules in build env: ```console Package Version ----------------------------- ----------- aiohappyeyeballs 2.4.3 aiohttp 3.10.10 aiosignal 1.3.1 alabaster 0.7.16 async-timeout 4.0.3 attrs 24.2.0 babel 2.16.0 bcrypt 4.2.0 build 1.2.2.post1 cffi 1.17.1 charset-normalizer 3.4.0 cryptography 42.0.8 defusedxml 0.7.1 distro 1.9.0 docutils 0.21.2 exceptiongroup 1.1.3 frozenlist 1.4.1 hatch-vcs 0.4.0 hatchling 1.25.0 imagesize 1.4.1 importlib_metadata 8.5.0 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.4 lz4 4.3.3 MarkupSafe 3.0.2 multidict 6.1.0 numpy 1.26.4 numpydoc 1.7.0 packaging 24.0 pandas 2.2.1 paramiko 3.5.0 pathspec 0.12.1 pluggy 1.5.0 ply 3.11 pycparser 2.22 pyftpdlib 1.5.7 Pygments 2.18.0 PyNaCl 1.5.0 pyproject_hooks 1.2.0 pytest 8.2.2 pytest-asyncio 0.23.8 pytest-mock 3.14.0 pytest-vcr 1.0.2 python-dateutil 2.9.0.post0 pytz 2024.2 PyYAML 6.0.2 requests 2.32.3 setuptools 75.1.0 setuptools-scm 8.1.0 snowballstemmer 2.2.0 Sphinx 8.1.3 sphinx_design 0.6.1 sphinxcontrib-applehelp 2.0.0 sphinxcontrib-devhelp 1.0.6 sphinxcontrib-htmlhelp 2.1.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 2.0.0 sphinxcontrib-serializinghtml 2.0.0 tabulate 0.9.0 tokenize_rt 6.0.0 tomli 2.0.1 tqdm 4.66.5 trove-classifiers 2024.10.17 typing_extensions 4.12.2 urllib3 2.2.2 vcrpy 6.0.2 wheel 0.44.0 wrapt 1.16.0 yarl 1.12.1 zipp 3.20.2 zstandard 0.22.0 ```

Please let me know if you need more details or want me to perform some diagnostics.

martindurant commented 1 month ago

Do you know how to test for the effect of -m "not network" within the test run session? All of these tests can be skipped for such a run.

kloczek commented 1 month ago

I don't see any of use network' pytest mark to skip some units so that module should not be affected by this. not notwork` should skip anything which requires more than localhost access. In this case looks like failing units uses access to localhost but nothing is listening.

martindurant commented 1 month ago

OK, so listening on localhost doesn't work; is there a way to make a pytest fixture that can know this restriction?