search

fsspec / filesystem_spec

A specification that python filesystems should adhere to.
BSD 3-Clause "New" or "Revised" License
1.03k stars 360 forks source link

2024.3.1: pytest fails in 5 `fsspec/implementations/tests/test_dbfs.py::test_dbfs*` units #1572

Closed kloczek closed 5 months ago

kloczek commented 6 months ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.3.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.3.1-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0 rootdir: /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1 configfile: pyproject.toml plugins: vcr-1.0.2, mock-3.14.0, asyncio-0.23.6 asyncio: mode=strict collected 1618 items / 8 skipped fsspec/implementations/tests/local/local_test.py ....................... [ 1%] ........................................................................ [ 5%] ........................................ [ 8%] fsspec/implementations/tests/memory/memory_test.py ..................... [ 9%] ........................................................................ [ 14%] .......................................... [ 16%] fsspec/implementations/tests/test_archive.py ........................... [ 18%] ..........................................................ssssssssssssss [ 22%] sss [ 22%] fsspec/implementations/tests/test_cached.py ............................ [ 24%] ..............ssssss............................... [ 27%] fsspec/implementations/tests/test_common.py ssss [ 28%] fsspec/implementations/tests/test_data.py .. [ 28%] fsspec/implementations/tests/test_dbfs.py FFFFFss [ 28%] fsspec/implementations/tests/test_dirfs.py ............................. [ 30%] ........................................................................ [ 34%] ...................... [ 36%] fsspec/implementations/tests/test_ftp.py ............. [ 37%] fsspec/implementations/tests/test_http.py .............................. [ 38%] ........................ [ 40%] fsspec/implementations/tests/test_libarchive.py s [ 40%] fsspec/implementations/tests/test_local.py .s........................... [ 42%] .s......................................................... [ 45%] fsspec/implementations/tests/test_memory.py ............................ [ 47%] [ 47%] fsspec/implementations/tests/test_reference.py ................s.....ss. [ 49%] .s [ 49%] fsspec/implementations/tests/test_sftp.py ssssssssssss [ 50%] fsspec/implementations/tests/test_tar.py ......................... [ 51%] fsspec/implementations/tests/test_webhdfs.py sssssssssss [ 52%] fsspec/implementations/tests/test_zip.py ......... [ 52%] fsspec/tests/test_api.py ...............x...... [ 54%] fsspec/tests/test_async.py .........s... [ 55%] fsspec/tests/test_caches.py ............................................ [ 57%] ........................................................................ [ 62%] ............................ [ 63%] fsspec/tests/test_callbacks.py ........ [ 64%] fsspec/tests/test_compression.py .....s [ 64%] fsspec/tests/test_config.py ....... [ 65%] fsspec/tests/test_core.py .............................................. [ 68%] ..s....... [ 68%] fsspec/tests/test_file.py ........... [ 69%] fsspec/tests/test_generic.py ...... [ 69%] fsspec/tests/test_mapping.py ................. [ 70%] fsspec/tests/test_parquet.py sssssssssssssssssssssssssssssssssssssssssss [ 73%] ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 77%] ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 82%] sssss [ 82%] fsspec/tests/test_registry.py .......s [ 83%] fsspec/tests/test_spec.py ....................x......................... [ 85%] ........................................................................ [ 90%] ........................................................................ [ 94%] .......................... [ 96%] fsspec/tests/test_utils.py ............................................. [ 99%] ............ [100%] =================================== FAILURES =================================== ____________________________ test_dbfs_file_listing ____________________________ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: > return complexjson.loads(self.text, **kwargs) /usr/lib/python3.10/site-packages/requests/models.py:969: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python3.10/json/__init__.py:346: in loads return _default_decoder.decode(s) /usr/lib64/python3.10/json/decoder.py:337: in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = s = '\x1f�\x08\x00\x00\x00\x00\x00\x04\x03��K\n�0\x10\x06�bҦ��\x01���F��6�`_$\x13\x17��ݪ\x0b���c7�\x0c\x1f̣�+�Ɓ>u�ft\x03\...��6��\x19ZQ���r1X�4}H����C)���j�hJƑ�r7Ҽ3v\x0e�\x19�\x12\x11�\x11�X�1�Ȭ3��\x17\x00\x00\x00��\x03\x00�ϭ:\x03\x03\x00\x00' idx = 0 def raw_decode(self, s, idx=0): """Decode a JSON document from ``s`` (a ``str`` beginning with a JSON document) and return a 2-tuple of the Python representation and the index in ``s`` where the document ended. This can be used to decode a JSON document from a string that may have extraneous data at the end. """ try: obj, end = self.scan_once(s, idx) except StopIteration as err: > raise JSONDecodeError("Expecting value", s, err.value) from None E json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib64/python3.10/json/decoder.py:355: JSONDecodeError During handling of the above exception, another exception occurred: dbfsFS = @pytest.mark.vcr() def test_dbfs_file_listing(dbfsFS): > assert "/FileStore" in dbfsFS.ls("/", detail=False) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/tests/test_dbfs.py:127: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:75: in ls r = self._send_to_api( /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:271: in _send_to_api return r.json() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: return complexjson.loads(self.text, **kwargs) except JSONDecodeError as e: # Catch JSON-related errors and raise as requests.JSONDecodeError # This aliases json.JSONDecodeError and simplejson.JSONDecodeError > raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) E requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib/python3.10/site-packages/requests/models.py:973: JSONDecodeError _______________________________ test_dbfs_mkdir ________________________________ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: > return complexjson.loads(self.text, **kwargs) /usr/lib/python3.10/site-packages/requests/models.py:969: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python3.10/json/__init__.py:346: in loads return _default_decoder.decode(s) /usr/lib64/python3.10/json/decoder.py:337: in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = s = '\x1f�\x08\x00\x00\x00\x00\x00\x04\x03��An� \x10E�2kW�`\x06�\x07�\x05��"kZHCk\x17\x0b&��(w/���Ra9\x12�\x7f�|�p��K0<_a!....C)�X�(�p�\x00�W�\x0b��2�t���x�����7���\x1f�^~{��"n=1}��\x16;�߾B��\x14�\x17+���\x07\x00\x00��\x03\x00�rccT\x03\x00\x00' idx = 0 def raw_decode(self, s, idx=0): """Decode a JSON document from ``s`` (a ``str`` beginning with a JSON document) and return a 2-tuple of the Python representation and the index in ``s`` where the document ended. This can be used to decode a JSON document from a string that may have extraneous data at the end. """ try: obj, end = self.scan_once(s, idx) except StopIteration as err: > raise JSONDecodeError("Expecting value", s, err.value) from None E json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib64/python3.10/json/decoder.py:355: JSONDecodeError During handling of the above exception, another exception occurred: dbfsFS = @pytest.mark.vcr() def test_dbfs_mkdir(dbfsFS): dbfsFS.rm("/FileStore/my", recursive=True) > assert "/FileStore/my" not in dbfsFS.ls("/FileStore/", detail=False) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/tests/test_dbfs.py:136: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:75: in ls r = self._send_to_api( /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:271: in _send_to_api return r.json() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: return complexjson.loads(self.text, **kwargs) except JSONDecodeError as e: # Catch JSON-related errors and raise as requests.JSONDecodeError # This aliases json.JSONDecodeError and simplejson.JSONDecodeError > raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) E requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib/python3.10/site-packages/requests/models.py:973: JSONDecodeError ___________________________ test_dbfs_write_and_read ___________________________ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: > return complexjson.loads(self.text, **kwargs) /usr/lib/python3.10/site-packages/requests/models.py:969: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python3.10/json/__init__.py:346: in loads return _default_decoder.decode(s) /usr/lib64/python3.10/json/decoder.py:337: in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = s = '\x1f�\x08\x00\x00\x00\x00\x00\x04\x03��An� \x10E�2kW�`\x06�\x07�\x05��"kZHCk\x17\x0b&��(w/���Ra9\x12�\x7f�|�p��K0<_a!....C)�X�(�p�\x00�W�\x0b��2�t���x�����7���\x1f�^~{��"n=1}��\x16;�߾B��\x14�\x17+���\x07\x00\x00��\x03\x00�rccT\x03\x00\x00' idx = 0 def raw_decode(self, s, idx=0): """Decode a JSON document from ``s`` (a ``str`` beginning with a JSON document) and return a 2-tuple of the Python representation and the index in ``s`` where the document ended. This can be used to decode a JSON document from a string that may have extraneous data at the end. """ try: obj, end = self.scan_once(s, idx) except StopIteration as err: > raise JSONDecodeError("Expecting value", s, err.value) from None E json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib64/python3.10/json/decoder.py:355: JSONDecodeError During handling of the above exception, another exception occurred: dbfsFS = @pytest.mark.vcr() def test_dbfs_write_and_read(dbfsFS): dbfsFS.rm("/FileStore/file.csv") > assert "/FileStore/file.csv" not in dbfsFS.ls("/FileStore/", detail=False) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/tests/test_dbfs.py:158: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:75: in ls r = self._send_to_api( /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:271: in _send_to_api return r.json() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: return complexjson.loads(self.text, **kwargs) except JSONDecodeError as e: # Catch JSON-related errors and raise as requests.JSONDecodeError # This aliases json.JSONDecodeError and simplejson.JSONDecodeError > raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) E requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib/python3.10/site-packages/requests/models.py:973: JSONDecodeError _____________________________ test_dbfs_read_range _____________________________ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: > return complexjson.loads(self.text, **kwargs) /usr/lib/python3.10/site-packages/requests/models.py:969: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python3.10/json/__init__.py:346: in loads return _default_decoder.decode(s) /usr/lib64/python3.10/json/decoder.py:337: in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = s = '\x1f�\x08\x00\x00\x00\x00\x00\x04\x03��An� \x10E�2kW�`\x06�\x07�\x05��"kZHCk\x17\x0b&��(w/���Ra9\x12�\x7f�|�p��K0<_a!....C)�X�(�p�\x00�W�\x0b��2�t���x�����7���\x1f�^~{��"n=1}��\x16;�߾B��\x14�\x17+���\x07\x00\x00��\x03\x00�rccT\x03\x00\x00' idx = 0 def raw_decode(self, s, idx=0): """Decode a JSON document from ``s`` (a ``str`` beginning with a JSON document) and return a 2-tuple of the Python representation and the index in ``s`` where the document ended. This can be used to decode a JSON document from a string that may have extraneous data at the end. """ try: obj, end = self.scan_once(s, idx) except StopIteration as err: > raise JSONDecodeError("Expecting value", s, err.value) from None E json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib64/python3.10/json/decoder.py:355: JSONDecodeError During handling of the above exception, another exception occurred: dbfsFS = @pytest.mark.vcr() def test_dbfs_read_range(dbfsFS): dbfsFS.rm("/FileStore/file.txt") > assert "/FileStore/file.txt" not in dbfsFS.ls("/FileStore/", detail=False) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/tests/test_dbfs.py:177: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:75: in ls r = self._send_to_api( /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:271: in _send_to_api return r.json() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: return complexjson.loads(self.text, **kwargs) except JSONDecodeError as e: # Catch JSON-related errors and raise as requests.JSONDecodeError # This aliases json.JSONDecodeError and simplejson.JSONDecodeError > raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) E requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib/python3.10/site-packages/requests/models.py:973: JSONDecodeError _________________________ test_dbfs_read_range_chunked _________________________ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: > return complexjson.loads(self.text, **kwargs) /usr/lib/python3.10/site-packages/requests/models.py:969: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python3.10/json/__init__.py:346: in loads return _default_decoder.decode(s) /usr/lib64/python3.10/json/decoder.py:337: in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = s = '\x1f�\x08\x00\x00\x00\x00\x00\x04\x03��An� \x10E�2kW�`\x06�\x07�\x05��"kZHCk\x17\x0b&��(w/���Ra9\x12�\x7f�|�p��K0<_a!....C)�X�(�p�\x00�W�\x0b��2�t���x�����7���\x1f�^~{��"n=1}��\x16;�߾B��\x14�\x17+���\x07\x00\x00��\x03\x00�rccT\x03\x00\x00' idx = 0 def raw_decode(self, s, idx=0): """Decode a JSON document from ``s`` (a ``str`` beginning with a JSON document) and return a 2-tuple of the Python representation and the index in ``s`` where the document ended. This can be used to decode a JSON document from a string that may have extraneous data at the end. """ try: obj, end = self.scan_once(s, idx) except StopIteration as err: > raise JSONDecodeError("Expecting value", s, err.value) from None E json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib64/python3.10/json/decoder.py:355: JSONDecodeError During handling of the above exception, another exception occurred: dbfsFS = @pytest.mark.vcr() def test_dbfs_read_range_chunked(dbfsFS): dbfsFS.rm("/FileStore/large_file.txt") > assert "/FileStore/large_file.txt" not in dbfsFS.ls("/FileStore/", detail=False) /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/tests/test_dbfs.py:190: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:75: in ls r = self._send_to_api( /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/implementations/dbfs.py:271: in _send_to_api return r.json() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , kwargs = {} def json(self, **kwargs): r"""Returns the json-encoded content of a response, if any. :param \*\*kwargs: Optional arguments that ``json.loads`` takes. :raises requests.exceptions.JSONDecodeError: If the response body does not contain valid json. """ if not self.encoding and self.content and len(self.content) > 3: # No encoding set. JSON RFC 4627 section 3 states we should expect # UTF-8, -16 or -32. Detect which one to use; If the detection or # decoding fails, fall back to `self.text` (using charset_normalizer to make # a best guess). encoding = guess_json_utf(self.content) if encoding is not None: try: return complexjson.loads(self.content.decode(encoding), **kwargs) except UnicodeDecodeError: # Wrong UTF codec detected; usually because it's not UTF-8 # but some other 8-bit codec. This is an RFC violation, # and the server didn't bother to tell us what codec *was* # used. pass except JSONDecodeError as e: raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) try: return complexjson.loads(self.text, **kwargs) except JSONDecodeError as e: # Catch JSON-related errors and raise as requests.JSONDecodeError # This aliases json.JSONDecodeError and simplejson.JSONDecodeError > raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) E requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) /usr/lib/python3.10/site-packages/requests/models.py:973: JSONDecodeError ================================== XFAILURES =================================== __________________________ test_multilevel_chained_fs __________________________ @pytest.mark.xfail(reason="see issue #334", strict=True) def test_multilevel_chained_fs(): """This test reproduces fsspec/filesystem_spec#334""" import zipfile d1 = tempfile.mkdtemp() f1 = os.path.join(d1, "f1.zip") with zipfile.ZipFile(f1, mode="w") as z: # filename, content z.writestr("foo.txt", "foo.txt") z.writestr("bar.txt", "bar.txt") # We expected this to be the correct syntax > with pytest.raises(IsADirectoryError): E Failed: DID NOT RAISE fsspec/tests/test_api.py:252: Failed __________________________________ test_find ___________________________________ @pytest.mark.xfail def test_find(): """Test .find() method on debian server (ftp, https) with constant folder""" filesystem, host, test_path = ( FTPFileSystem, "ftp.fau.de", "ftp://ftp.fau.de/debian-cd/current/amd64/log/success", ) test_fs = filesystem(host) filenames_ftp = test_fs.find(test_path) > assert filenames_ftp E assert [] fsspec/tests/test_spec.py:699: AssertionError =============================== warnings summary =============================== fsspec/tests/test_async.py::test_run_coros_in_chunks /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/tests/test_async.py:98: RuntimeWarning: coroutine 'test_run_coros_in_chunks..runner' was never awaited await asyncio.sleep(0) Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. fsspec/tests/test_async.py::test_async_streamed_file_write /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.3.1/fsspec/asyn.py:937: RuntimeWarning: coroutine 'test_run_coros_in_chunks..runner' was never awaited mth = sync_wrapper(getattr(obj, method), obj=obj) Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ SKIPPED [1] fsspec/implementations/tests/test_arrow.py:5: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_dask.py:5: could not import 'distributed': No module named 'distributed' SKIPPED [1] fsspec/implementations/tests/test_git.py:11: could not import 'pygit2': No module named 'pygit2' SKIPPED [1] fsspec/implementations/tests/test_jupyter.py:10: could not import 'notebook': No module named 'notebook' SKIPPED [1] fsspec/implementations/tests/test_smb.py:14: could not import 'smbprotocol': No module named 'smbprotocol' SKIPPED [1] fsspec/tests/test_downstream.py:3: could not import 's3fs': No module named 's3fs' SKIPPED [1] fsspec/tests/test_fuse.py:9: could not import 'fuse': No module named 'fuse' SKIPPED [1] fsspec/tests/test_gui.py:3: could not import 'panel': No module named 'panel' SKIPPED [18] fsspec/implementations/tests/test_archive.py:45: could not import 'libarchive': No module named 'libarchive' SKIPPED [6] fsspec/implementations/tests/test_cached.py:874: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_common.py:10: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [3] fsspec/implementations/tests/test_common.py:21: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_dbfs.py:200: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_dbfs.py:228: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_local.py:106: could not import 's3fs': No module named 's3fs' SKIPPED [1] fsspec/implementations/tests/test_local.py:260: could not import 'cloudpickle': No module named 'cloudpickle' SKIPPED [1] fsspec/implementations/tests/test_reference.py:392: could not import 'zarr': No module named 'zarr' SKIPPED [1] fsspec/implementations/tests/test_reference.py:538: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_reference.py:584: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_reference.py:692: could not import 'zarr': No module named 'zarr' SKIPPED [1] fsspec/implementations/tests/test_sftp.py:69: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:84: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:102: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:139: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:149: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:181: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:203: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:220: docker run not available SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:43: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:49: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:57: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:67: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:84: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:100: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:121: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:140: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:153: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:163: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:177: htcluster not found SKIPPED [1] fsspec/tests/test_async.py:134: only for windows SKIPPED [1] fsspec/tests/test_compression.py:134: could not import 'snappy': No module named 'snappy' SKIPPED [1] fsspec/tests/test_core.py:338: could not import 's3fs': No module named 's3fs' SKIPPED [64] fsspec/tests/test_parquet.py:37: fastparquet not found SKIPPED [64] fsspec/tests/test_parquet.py:37: pyarrow not found SKIPPED [64] fsspec/tests/test_parquet.py:37: No parquet engine (fastparquet or pyarrow) found SKIPPED [1] fsspec/tests/test_registry.py:135: could not import 's3fs': No module named 's3fs' XFAIL fsspec/tests/test_api.py::test_multilevel_chained_fs - see issue #334 XFAIL fsspec/tests/test_spec.py::test_find FAILED fsspec/implementations/tests/test_dbfs.py::test_dbfs_file_listing - re... FAILED fsspec/implementations/tests/test_dbfs.py::test_dbfs_mkdir - requests.... FAILED fsspec/implementations/tests/test_dbfs.py::test_dbfs_write_and_read - ... FAILED fsspec/implementations/tests/test_dbfs.py::test_dbfs_read_range - requ... FAILED fsspec/implementations/tests/test_dbfs.py::test_dbfs_read_range_chunked = 5 failed, 1356 passed, 263 skipped, 2 xfailed, 2 warnings in 96.24s (0:01:36) = ```
List of installed modules in build env: ```console Package Version ----------------------------- ----------- aiohttp 3.9.3 aiosignal 1.3.1 alabaster 0.7.16 async-timeout 4.0.3 attrs 23.2.0 Babel 2.14.0 bcrypt 4.1.2 build 1.2.1 cffi 1.16.0 charset-normalizer 3.3.2 cryptography 42.0.5 docutils 0.20.1 exceptiongroup 1.1.3 frozenlist 1.4.1 idna 3.7 imagesize 1.4.1 importlib_metadata 7.1.0 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.3 lz4 4.3.3 MarkupSafe 2.1.5 multidict 6.0.5 numpy 1.26.5 numpydoc 1.7.0 packaging 24.0 pandas 2.2.1 paramiko 3.4.0 pluggy 1.4.0 ply 3.11 pycparser 2.22 pyftpdlib 1.5.7 Pygments 2.17.2 PyNaCl 1.5.0 pyproject_hooks 1.0.0 pytest 8.1.1 pytest-asyncio 0.23.6 pytest-mock 3.14.0 pytest-vcr 1.0.2 python-dateutil 2.9.0.post0 pytz 2024.1 PyYAML 6.0.1 requests 2.31.0 setuptools 69.4.0 snowballstemmer 2.2.0 Sphinx 7.2.6 sphinx_design 0.5.0 sphinxcontrib-applehelp 1.0.8 sphinxcontrib-devhelp 1.0.6 sphinxcontrib-htmlhelp 2.0.5 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.7 sphinxcontrib-serializinghtml 1.1.10 tabulate 0.9.0 tokenize_rt 5.2.0 tomli 2.0.1 tqdm 4.66.2 typing_extensions 4.11.0 urllib3 2.2.1 vcrpy 6.0.1 wheel 0.43.0 wrapt 1.16.0 yarl 1.9.4 zipp 3.18.1 zstandard 0.22.0 ```

Please let me know if you need more details or want me to perform some diagnostics.

martindurant commented 5 months ago

The DBFS tests use HTTP mocking to work. That should be OK when there is no network, but apparently there is something amiss. I'd be happy to simply skip these tests in such a situation; do you happen to have an idea of how to best detect from within the python test module that HTTP is likely to fail?

kloczek commented 5 months ago

I've tested that on the box with full access to public network and fails are exactly the same.

martindurant commented 5 months ago

Thanks for looking into it, @kloczek ; I'll try to figure out what's going on.

martindurant commented 5 months ago

If you have a moment: can you please check if the YAML files fsspec/implementations/tests/cassettes/test_dbfs/test_dbfs*.yaml exist in the test install?

kloczek commented 5 months ago

In case of your module on packaging I'm using tar ball autogenerated from git tag in which those yaml files are included.. BTW it would be good to move fsspec/implementations/tests/ and fsspec/tests/ to tests/ to avoid include those files in generated .whl because now test suite is part of the installed resources.

martindurant commented 5 months ago

https://github.com/fsspec/filesystem_spec/pull/1611 discussed removing test files (this was an artefact of moving the build system to hatch)

kloczek commented 5 months ago

Easiest way to do that separation is move all to tests/

kloczek commented 5 months ago

Just tested 2024.6.0 and looks like pytest is OK now with few warnigs

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.6.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.6.0-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.6.0 configfile: pyproject.toml plugins: hypothesis-6.100.0, mock-3.14.0, examples-0.0.10, benchmark-4.0.0, Faker-25.3.0, vcr-1.0.2, asyncio-0.23.7 asyncio: mode=strict collected 1701 items / 9 skipped fsspec/implementations/tests/local/local_test.py .................................................................................................................................... [ 7%] ... [ 7%] fsspec/implementations/tests/memory/memory_test.py .................................................................................................................................. [ 15%] ..... [ 15%] fsspec/implementations/tests/test_archive.py .....................................................................................sssssssssssssssss [ 21%] fsspec/implementations/tests/test_cached.py ..........................................ssssssssssss............................... [ 26%] fsspec/implementations/tests/test_common.py ssss [ 27%] fsspec/implementations/tests/test_data.py .. [ 27%] fsspec/implementations/tests/test_dirfs.py ........................................................................................................................... [ 34%] fsspec/implementations/tests/test_ftp.py ............. [ 35%] fsspec/implementations/tests/test_http.py ....................................................... [ 38%] fsspec/implementations/tests/test_libarchive.py s [ 38%] fsspec/implementations/tests/test_local.py .s.........................................................................................ss........ss.sssss.......sss....s.............. [ 46%] ............ [ 47%] fsspec/implementations/tests/test_memory.py .............................. [ 49%] fsspec/implementations/tests/test_reference.py .................s.....ss..s [ 50%] fsspec/implementations/tests/test_sftp.py ssssssssssss [ 51%] fsspec/implementations/tests/test_tar.py ......................... [ 52%] fsspec/implementations/tests/test_webhdfs.py sssssssssss [ 53%] fsspec/implementations/tests/test_zip.py ......... [ 54%] fsspec/tests/test_api.py ...............x...... [ 55%] fsspec/tests/test_async.py .........s... [ 56%] fsspec/tests/test_caches.py ................................................................................................................................................. [ 64%] fsspec/tests/test_callbacks.py ........ [ 65%] fsspec/tests/test_compression.py .....s [ 65%] fsspec/tests/test_config.py ....... [ 65%] fsspec/tests/test_core.py .................................................s....... [ 69%] fsspec/tests/test_file.py ........... [ 69%] fsspec/tests/test_generic.py ...... [ 70%] fsspec/tests/test_mapping.py ................. [ 71%] fsspec/tests/test_parquet.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 80%] ssssssssssssssssssssssssssssssssssssssss [ 82%] fsspec/tests/test_registry.py ......s [ 82%] fsspec/tests/test_spec.py ....................x...................................................................................................................................... [ 92%] ................................................................. [ 95%] fsspec/tests/test_utils.py ...................................................................... [100%] ========================================================================================= XFAILURES ========================================================================================= ________________________________________________________________________________ test_multilevel_chained_fs _________________________________________________________________________________ @pytest.mark.xfail(reason="see issue #334", strict=True) def test_multilevel_chained_fs(): """This test reproduces fsspec/filesystem_spec#334""" import zipfile d1 = tempfile.mkdtemp() f1 = os.path.join(d1, "f1.zip") with zipfile.ZipFile(f1, mode="w") as z: # filename, content z.writestr("foo.txt", "foo.txt") z.writestr("bar.txt", "bar.txt") # We expected this to be the correct syntax > with pytest.raises(IsADirectoryError): E Failed: DID NOT RAISE fsspec/tests/test_api.py:252: Failed _________________________________________________________________________________________ test_find _________________________________________________________________________________________ @pytest.mark.xfail def test_find(): """Test .find() method on debian server (ftp, https) with constant folder""" filesystem, host, test_path = ( FTPFileSystem, "ftp.fau.de", "ftp://ftp.fau.de/debian-cd/current/amd64/log/success", ) test_fs = filesystem(host) filenames_ftp = test_fs.find(test_path) > assert filenames_ftp E assert [] fsspec/tests/test_spec.py:699: AssertionError ===================================================================================== warnings summary ====================================================================================== fsspec/tests/test_async.py::test_run_coros_in_chunks /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.6.0/fsspec/asyn.py:267: RuntimeWarning: coroutine 'test_run_coros_in_chunks..runner' was never awaited done, pending = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED) Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. fsspec/tests/test_caches.py::test_cache_getitem[background] /usr/lib64/python3.10/concurrent/futures/thread.py:151: RuntimeWarning: coroutine 'test_run_coros_in_chunks..runner' was never awaited self._idle_semaphore = threading.Semaphore(0) Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================== short test summary info ================================================================================== SKIPPED [1] fsspec/implementations/tests/test_arrow.py:5: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_dask.py:5: could not import 'distributed': No module named 'distributed' SKIPPED [1] fsspec/implementations/tests/test_dbfs.py:33: These tests need to be re-recorded. SKIPPED [1] fsspec/implementations/tests/test_git.py:11: could not import 'pygit2': No module named 'pygit2' SKIPPED [1] fsspec/implementations/tests/test_jupyter.py:10: could not import 'notebook': No module named 'notebook' SKIPPED [1] fsspec/implementations/tests/test_smb.py:15: could not import 'smbprotocol': No module named 'smbprotocol' SKIPPED [1] fsspec/tests/test_downstream.py:3: could not import 's3fs': No module named 's3fs' SKIPPED [1] fsspec/tests/test_fuse.py:9: could not import 'fuse': No module named 'fuse' SKIPPED [1] fsspec/tests/test_gui.py:3: could not import 'panel': No module named 'panel' SKIPPED [18] fsspec/implementations/tests/test_archive.py:45: could not import 'libarchive': No module named 'libarchive' SKIPPED [6] fsspec/implementations/tests/test_cached.py:874: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [6] fsspec/implementations/tests/test_cached.py:888: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_common.py:10: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [3] fsspec/implementations/tests/test_common.py:21: could not import 'pyarrow.fs': No module named 'pyarrow' SKIPPED [1] fsspec/implementations/tests/test_local.py:135: could not import 's3fs': No module named 's3fs' SKIPPED [2] fsspec/implementations/tests/test_local.py:785: Windows only SKIPPED [2] fsspec/implementations/tests/test_local.py:801: Windows only SKIPPED [5] fsspec/implementations/tests/test_local.py:823: Windows only SKIPPED [3] fsspec/implementations/tests/test_local.py:880: Windows only SKIPPED [1] fsspec/implementations/tests/test_local.py:895: Windows only SKIPPED [1] fsspec/implementations/tests/test_reference.py:433: could not import 'zarr': No module named 'zarr' SKIPPED [1] fsspec/implementations/tests/test_reference.py:579: could not import 'fastparquet': No module named 'fastparquet' SKIPPED [1] fsspec/implementations/tests/test_reference.py:625: could not import 'fastparquet': No module named 'fastparquet' SKIPPED [1] fsspec/implementations/tests/test_reference.py:733: could not import 'zarr': No module named 'zarr' SKIPPED [1] fsspec/implementations/tests/test_sftp.py:71: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:86: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:104: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:143: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:152: docker run not available SKIPPED [1] fsspec/implementations/tests/test_sftp.py:183: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:205: docker run not available SKIPPED [2] fsspec/implementations/tests/test_sftp.py:222: docker run not available SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:43: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:49: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:57: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:67: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:84: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:100: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:121: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:140: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:153: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:163: htcluster not found SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:177: htcluster not found SKIPPED [1] fsspec/tests/test_async.py:134: only for windows SKIPPED [1] fsspec/tests/test_compression.py:134: could not import 'snappy': No module named 'snappy' SKIPPED [1] fsspec/tests/test_core.py:357: could not import 's3fs': No module named 's3fs' SKIPPED [64] fsspec/tests/test_parquet.py:37: fastparquet not found SKIPPED [64] fsspec/tests/test_parquet.py:37: pyarrow not found SKIPPED [64] fsspec/tests/test_parquet.py:37: No parquet engine (fastparquet or pyarrow) found SKIPPED [1] fsspec/tests/test_registry.py:129: could not import 's3fs': No module named 's3fs' XFAIL fsspec/tests/test_api.py::test_multilevel_chained_fs - see issue #334 XFAIL fsspec/tests/test_spec.py::test_find ============================================================ 1428 passed, 280 skipped, 2 xfailed, 2 warnings in 91.23s (0:01:31) ============================================================ /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-87593477-5596-402e-a1d6-2cb0ce4b0bf0/test_safe_get_no_perms0 : [Errno 39] Directory not empty: 'test_safe_get_no_perms0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-87593477-5596-402e-a1d6-2cb0ce4b0bf0/test_safe_set_no_perms0 : [Errno 39] Directory not empty: 'test_safe_set_no_perms0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-87593477-5596-402e-a1d6-2cb0ce4b0bf0/test_safe_delete_no_perms0 : [Errno 39] Directory not empty: 'test_safe_delete_no_perms0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-87593477-5596-402e-a1d6-2cb0ce4b0bf0/test_rmtree_errorhandler_rerai0 : [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-87593477-5596-402e-a1d6-2cb0ce4b0bf0 : [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-87593477-5596-402e-a1d6-2cb0ce4b0bf0' warnings.warn( ```
martindurant commented 5 months ago

OK, weird :) Thanks for trying the new version.