danielfrg / s3contents

Jupyter Notebooks in S3 - Jupyter Contents Manager implementation
Apache License 2.0
248 stars 88 forks source link

Wrong path for S3 stored notebooks #169

Closed djuarezg closed 1 year ago

djuarezg commented 1 year ago

When using relative and absolute paths of notebooks inside the S3 folder, it always reports wrongly that they belong to the root dir:

image

from s3contents import S3ContentsManager
from hybridcontents import HybridContentsManager
from jupyter_server.services.contents.largefilemanager import LargeFileManager
c = get_config()

c.NotebookApp.contents_manager_class = HybridContentsManager
c.NotebookApp.notebook_dir = "/home/jovyan/work"

c.HybridContentsManager.manager_classes = {
    # Associate the root directory with an S3ContentsManager.
    # This manager will receive all requests that don"t fall under any of the
    # other managers.
    "s3-shared": S3ContentsManager,
    # Associate /local_directory with a LargeFileManager.
    "": LargeFileManager,
}

c.HybridContentsManager.manager_kwargs = {
    # Args for root S3ContentsManager.
    "s3-shared": {
        "access_key_id": "asdasda",
        "secret_access_key": "sasdasd",
        "bucket": "asasdadsad",
        "endpoint_url": "asdasdasd",
        "skip_tls_verify": True,
        "prefix": "",
    },
    # Args for the LargeFileManager mapped to anything else but s3 shared content
    "": {
        "root_dir": "/home/jovyan/work",
    },
}

c.ContentsManager.files_handler_class = 'notebook.files.handlers.FilesHandler'
# these two do not make a difference
c.ContentsManager.root_dir = '/home/jovyan/work'
c.FileContentsManager.root_dir = '/home/jovyan/work'

Is there any way for files to report the correct path?

djuarezg commented 1 year ago

@danielfrg sorry to bother you but are you aware of any solution/workaround for such a use case? It does not seem to be a wild idea. Thanks!

EDIT: I think what I am trying to accomplish which is treating the S3 stored files as if they were locally present is not possible...

danielfrg commented 1 year ago

Hi, that is not possible. If you are trying to read a file from S3 you need to use something like boto to read it. Or mount the S3 bucket to the instance where you are a running the notebook server.