feat(pins): add support for custom storage options in `board_s3`

rstudio / pins-python

https://rstudio.github.io/pins-python/

MIT License

52 stars 12 forks source link

feat(pins): add support for custom storage options in `board_s3` #237

Closed ericmjl closed 4 months ago

ericmjl commented 4 months ago

Introduce the ability to pass arbitrary storage options to the underlying fsspec S3FileSystem in board_s3.
This enhancement allows for greater flexibility when connecting to S3-compatible services by enabling the specification of custom credentials, endpoints, and other S3FileSystem parameters.
Added documentation and examples to illustrate how to use the new storage_options parameter, including an example for connecting to Backblaze B2.

This change enables users to more easily integrate with a variety of S3-compatible storage solutions, improving the library's versatility and user experience.

Address issue: #235

Tag @isabelizimm!

ericmjl commented 4 months ago

@isabelizimm @machow I think I'm running into an issue with the docs, but I'm not quite sure how to address the issue, as it uses quartodoc (which I'm happy to learn, but may need help getting up-to-speed on).

In case it's helpful, this is a direct link to the exact line that is failing.

isabelizimm commented 4 months ago

Ah thank you so much for putting up this PR! Let me peek at the docs failures and run the tests on other backends to see if something jumps out 👀

update: These tests + docs failures should be fixed in #242; needed a bit of maintenance for the latest fsspec version that reordered some protocol names 👍

isabelizimm commented 4 months ago

One small nit, but everything else looks great! 😄

ericmjl commented 4 months ago

Woot! Looks like all of the non-skipped stuff passed :smile:.

ericmjl commented 4 months ago

Okie dokes, just pushed the requested change, @isabelizimm!