search

developmentseed / obstore

Simple, fast integration with Amazon S3, Google Cloud Storage, Azure Storage, and S3-compliant APIs like Cloudflare R2
https://developmentseed.org/obstore
MIT License
146 stars 3 forks source link

Support range in GetOptions #40

Closed kylebarron closed 4 weeks ago

kylebarron commented 4 weeks ago

Closes https://github.com/developmentseed/object-store-rs/issues/10

kylebarron commented 4 weeks ago

From https://github.com/zarr-developers/zarr-python/pull/1661#issuecomment-2433392608:

Note that they can can be negative, so a suffix range would be (-N, None).

@martindurant for now this PR uses (None, N) for a suffix request, where N is positive. This is documented in the docstring.

We may change this in the future to be a dict, with keys like offset, bounded, and suffix, to more closely match the upstream type: https://docs.rs/object_store/latest/object_store/enum.GetRange.html

martindurant commented 4 weeks ago

for now this PR uses (None, N)

That looks a lot like "first N bytes" to me, not suffix :| I could be biased.

kylebarron commented 4 weeks ago

I think in general it's too easy to get confused by these. But there's no reason to use (None, N) when you can easily by explicit about (0, N).