bioio-devs / bioio-ome-zarr

A BioIO reader plugin for reading Zarr files in the OME format.
BSD 3-Clause "New" or "Revised" License
0 stars 2 forks source link

Public s3 prefix without auth #26

Closed pgarrison closed 1 month ago

pgarrison commented 1 month ago

Trying again to do #25 but with the tests fixed this time!

Context

Previously bioio could not read from a public s3:// URL without AWS authentication due to this ome-zarr issue. As of 0.9.0 the issue is resolved.

Changes

Testing

pdm run pytest bioio_ome_zarr/tests/test_s3_read.py

pgarrison commented 1 month ago

Draft PR until #24 is merged.

pgarrison commented 1 month ago

Previously Dan asked:

LGTM. So if someone wants BioImage(zarr_s3_url), do they really have to do BioImage(zarr_s3_url, anon=True) ?

And I answered:

They must either provide AWS credentials or set anon=True. I agree the usability could be improved because setting anon=True for the same with an https URL will cause an error. I believe this should be the case not just for OME ZARRs but for any file read from a public s3:// path, since I expect all our readers use s3fs under the hood.

So if we wanted to improve the usability I think it happens in bioio or bioio-base by wrapping the BioImage __init__ logic with something like this:

try:
    # __init__ with user's fs_kwargs
except SomethingSpecific as e:
    if protocol == "s3://":
        # __init__ with user's fs_kwargs plus {anon: True}
    else:
        raise e