search

IBM / jupyterlab-s3-browser

A JupyterLab extension for browsing S3-compatible object storage
Apache License 2.0
120 stars 43 forks source link

Viewing a bucket with a large number of files/prefixes causes the extension to choke #7

Closed reevejd closed 4 years ago

reevejd commented 4 years ago

Viewing a bucket with 500+ (exact number required to reproduce is unknown) items causes the extension to reliably choke.

tomcatlingcma commented 4 years ago

+1 to this issue. Would love to use this extension but it gets destroyed by our bucket, which has millions of objects.

I think this could probably be solved by using 

objects = list(bucket.objects.filter(Prefix=path, Delimiter='/'))

at https://github.com/IBM/jupyterlab-s3-browser/blob/6c9693803650864c3922c76b0413d54b42e93825/jupyterlab_s3_browser/__init__.py#L181

I'll try to confirm and make a PR

brl0 commented 4 years ago

In addition to the extension choking, it seems to choke an entire jupyterlab session.

reevejd commented 4 years ago

I believe this was fixed in https://github.com/IBM/jupyterlab-s3-browser/pull/14, but please reopen if you're still having issues.