danielfrg / s3contents

Jupyter Notebooks in S3 - Jupyter Contents Manager implementation
Apache License 2.0
248 stars 88 forks source link

fsspec==0.9.0 breaks current s3contents #118

Closed lydian closed 3 years ago

lydian commented 3 years ago

When bumpping to fsspec==0.9.0, i found that s3contents constantly unable to find contents in the folder. (It keeps saying 404). Downpin to the previous version (0.8.7) seems to fix the issue! Wondering if anyone also having the same issue as well?

danielfrg commented 3 years ago

Definitely possible, i keep having to pin and unpin those fsspec and related libs dependencies :S

milutz commented 3 years ago

... adding to the fun dask/gcsfs 8.0 and dask/s3fs 0.6.0 ( both released 2 days ago) want to use fsspec.utils setup_logging which only became available in 0.9.0 (not in 0.8.7)

gth158a commented 3 years ago

After uninstalling 0.9.0 and installing 0.8.7 I get that same setup_logging error when launching jupyter lab

> pip uninstall fsspec
> pip install fsspec==0.8.7

> jupyter lab
[E 18:15:50.619 LabApp] Exception while loading config file /home/hadoop/.jupyter/jupyter_notebook_config.py
    Traceback (most recent call last):
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/traitlets/config/application.py", line 562, in _load_config_files
        config = loader.load_config()
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/traitlets/config/loader.py", line 457, in load_config
        self._read_file_as_dict()
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/traitlets/config/loader.py", line 489, in _read_file_as_dict
        py3compat.execfile(conf_filename, namespace)
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/ipython_genutils/py3compat.py", line 198, in execfile
        exec(compiler(f.read(), fname, 'exec'), glob, loc)
      File "/home/hadoop/.jupyter/jupyter_notebook_config.py", line 1, in <module>
        from s3contents import S3ContentsManager
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/s3contents/__init__.py", line 24, in <module>
        from .gcsmanager import GCSContentsManager  # noqa
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/s3contents/gcsmanager.py", line 1, in <module>
        from s3contents.gcs_fs import GCSFS
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/s3contents/gcs_fs.py", line 3, in <module>
        import gcsfs
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/gcsfs/__init__.py", line 5, in <module>
        from .core import GCSFileSystem
      File "/home/hadoop/anaconda3/envs/seabright-rook/lib/python3.6/site-packages/gcsfs/core.py", line 33, in <module>
        from fsspec.utils import stringify_path, setup_logging
    ImportError: cannot import name 'setup_logging'
gth158a commented 3 years ago

I downgraded gcsfs form 0.8.0 which uses 'setup_logging' and was released 3 days ago to 0.7.2 and now things work as before

> pip uninstall gcsfs
> pip install gcsfs==0.7.2

Bottomline freeze verison of fsspec==0.8.7 and gcsfs==0.7.2

danielfrg commented 3 years ago

Fixed on https://github.com/danielfrg/s3contents/pull/119

I just released 0.6.3 with that PR so we should be good for now :)

Thanks @gth158a for the PR and everyone else for reporting.