danielfrg / s3contents

Jupyter Notebooks in S3 - Jupyter Contents Manager implementation
Apache License 2.0
248 stars 88 forks source link

Extend support to notebook 7 #167

Closed antoine-galataud closed 1 year ago

antoine-galataud commented 1 year ago

Jupyter Notebook 7 is about to be released (as of writing, b3 released).

In this new version, most of the backend is based on Jupyter server. While https://github.com/danielfrg/s3contents/issues/161 has solved some compatibility issues, some problem remains.

For instance, there are still some notebook.* imports in ipycompat.py that raise exceptions on startup:

[E 2023-06-09 09:05:17.328 JupyterNotebookApp] Exception while loading config file /home/jupyter/.jupyter/jupyter_notebook_config.py
    Traceback (most recent call last):
      File "/home/jupyter/.local/lib/python3.10/site-packages/traitlets/config/application.py", line 909, in _load_config_files
        config = loader.load_config()
      File "/home/jupyter/.local/lib/python3.10/site-packages/traitlets/config/loader.py", line 626, in load_config
        self._read_file_as_dict()
      File "/home/jupyter/.local/lib/python3.10/site-packages/traitlets/config/loader.py", line 659, in _read_file_as_dict
        exec(compile(f.read(), conf_filename, "exec"), namespace, namespace)  # noqa
      File "/home/jupyter/.jupyter/jupyter_notebook_config.py", line 2, in <module>
        from s3contents import S3ContentsManager
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/__init__.py", line 3, in <module>
        from .s3manager import S3ContentsManager  # noqa
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/s3manager.py", line 6, in <module>
        from s3contents.genericmanager import GenericContentsManager, from_dict
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/genericmanager.py", line 17, in <module>
        from s3contents.genericfs import GenericFSError, NoSuchFile
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/genericfs.py", line 5, in <module>
        from s3contents.ipycompat import HasTraits
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/ipycompat.py", line 18, in <module>
        from notebook.services.contents.checkpoints import (
    ModuleNotFoundError: No module named 'notebook.services'

I will PR a change that I've just tested to make it compatible with Notebook 7.

antoine-galataud commented 1 year ago

In addition, when you enable jupyter-collaboration extension for real time collaboration, opening a notebook will raise a JSON parsing exception such as following:

[E 2023-06-09 13:22:32.957 ServerApp] Error initializing: notebook.ipynb
    NotJSONError("Notebook does not appear to be JSON: 'eyJtZXRhZGF0YSI6IHsia2VybmVsc3BlYyI6IHs...")
    Traceback (most recent call last):
      File "/home/jupyter/.local/lib/python3.10/site-packages/nbformat/reader.py", line 20, in parse_json
        nb_dict = json.loads(s, **kwargs)
      File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
      File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
        raise JSONDecodeError("Expecting value", s, err.value) from None
    json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/home/jupyter/.local/lib/python3.10/site-packages/jupyter_collaboration/handlers.py", line 167, in open
        await self.room.initialize()
      File "/home/jupyter/.local/lib/python3.10/site-packages/jupyter_collaboration/rooms.py", line 98, in initialize
        model = await self._file.load_content(self._file_format, self._file_type, True)
      File "/home/jupyter/.local/lib/python3.10/site-packages/jupyter_collaboration/loaders.py", line 115, in load_content
        self._contents_manager.get(
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/genericmanager.py", line 135, in get
        return func(path=path, content=content, format=format)
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/genericmanager.py", line 154, in _get_notebook
        return self._notebook_model_from_path(
      File "/home/jupyter/.local/lib/python3.10/site-packages/s3contents/genericmanager.py", line 295, in _notebook_model_from_path
        nb_content = reads(file_content, as_version=NBFORMAT_VERSION)
      File "/home/jupyter/.local/lib/python3.10/site-packages/nbformat/__init__.py", line 89, in reads
        nb = reader.reads(s, **kwargs)
      File "/home/jupyter/.local/lib/python3.10/site-packages/nbformat/reader.py", line 76, in reads
        nb_dict = parse_json(s, **kwargs)
      File "/home/jupyter/.local/lib/python3.10/site-packages/nbformat/reader.py", line 26, in parse_json
        raise NotJSONError(message) from e
    nbformat.reader.NotJSONError: Notebook does not appear to be JSON: 'eyJtZXRhZGF0YSI6IHsia2VybmVsc3BlYyI6IHs...

This was tested only with AWS S3 as a storage backend. The fix hence only focuses on s3_fs.