Closed jagane-opensource closed 4 years ago
Hello James,
Can you provide me access to the bucket that you are seeing problems with? I can test it myself. Otherwise, if you can add some more detail regarding the problems you are seeing, that would be good. I will chase the problem down. Thanks
I'm not sure how to securely grant you access in a way that would allow you to browse my bucket using this jupyterlab extension, but I hope this is enough to reproduce. I've created a new bucket with a single object with the key prefix1/prefix2/test.txt
When I try to access it via this extension I get this far:
And then when I click on prefix1
I get:
James - how are you creating these 'directories' or prefixes? I have found a peculiarity in the way Amazon S3 deals with fake directories. If I create these fake directories using the S3 web console and then list them, the S3 REST call list_objects_v2 returns a dummy entry that does not serve us well. So I filter out these fake directory entries and that seems to work well.
I have now tested using two different methods: Method 1: Creating fake dirs using the S3 web console and then copying a file into a directory several levels deep Method 2: By copying a file directly into a several level deep hierarchy. e.g. aws s3 cp onefile.txt s3://testbucket/testlevel1/testlevel2/testlevel3/onefile.txt
Both of the above methods work now. I am still not convinced that I have solved the problem you are seeing - it could some other bug entirely. Can you test with the new change? Also, can you describe how you created the directories prefix1 and prefix2 in your example above?
This set of changes does the following: