Closed blairdrummond closed 4 years ago
It seems to not play nice with folders within buckets in general? Does it expect a flat file layout?
Thanks for opening the issue. It's supposed to work for any file layout. I'll take a look later today.
I'm able to reproduce the issue. Unless I'm mistaken it's minio specific, which is surprising. I fully intend for this extension to support minio so I'll continue looking into it. Thanks again for raising the issue.
Yeah I did run this on minio; I should have mentioned that!
I was able tools recreate the issue using a COS instance on IBM Cloud:
s3://mybucket/
)s3://mybucket/prefix1/object1
, s3://mybucket/prefix2/prefix3/object2
)prefix1
or prefix2
) the key error is raised. Looking at the code it appears that during that operation a call is made to s3client.get_object(Bucket=bucket_name, Key=path)
e.g. s3client.get_object(Bucket=mybucket, Key=prefix1)
, which triggers the 404/not found.
Digging a bit more into the code get_s3_objects_from_path(s3, path)
in __init__.py
appears to treat path
as a "directory" if it ends with /
. For some unknown reason (I haven't look at the labextension) the trailing slash is never present if path refers to a key prefix, e.g. prefix1
. Therefore these requests always result in a do_get_object
call instead of a do_list_objects_v2
call.
Hope this helps!
Thanks for the info. I believe this has now been fixed in the most recent releases for JupyterLab 1.x and 2.x. Please reopen this issue if you continue to encounter it after updating. Thanks!
Still seeing this Amazon S3 and
jupyterlab==2.2.2 jupyterlab-s3-browser==0.6.1
I can provide more info if there is something that would help.
@jarutis what version of the labextension are you using? You can run jupyter labextension list
to show the versions of all the labextensions you have installed.
Here they are:
JupyterLab v2.2.2
Known labextensions:
app dir: /home/ubuntu/conda/share/jupyter/lab
@jupyter-voila/jupyterlab-preview v1.1.0 enabled OK
@jupyter-widgets/jupyterlab-manager v2.0.0 enabled OK
@jupyterlab/git v0.20.0 enabled OK
@krassowski/jupyterlab-lsp v1.1.2 enabled OK
@ryantam626/jupyterlab_code_formatter v1.3.6 enabled OK
jupyter-matplotlib v0.7.3 enabled OK
jupyterlab-s3-browser v0.4.1 enabled OK
nbdime-jupyterlab v2.0.0 enabled OK
Sorry for the delayed response. if you update from version 0.4.1
to 0.6.1
it should resolve the issue. If not, please let me know.
Seems idiosyncratic, but on some folders, I get this error
However the folder is not empty, I can see the contents in minio. I don't have any fancy policies, though I don't know this tech well. Error on the notebook server is below
As I said, the folders definitely exist