search

data-engineering-collective / minimalkv

A minimal key-value store interface for binary data (maintained fork of simplekv).
https://minimalkv.readthedocs.io/en/latest/index.html
Other
17 stars 9 forks source link

Use fsspec for store defintion #47

Open janjagusch opened 1 year ago

janjagusch commented 1 year ago

Many stores, like GoogleCloudStore, are written from scratch. It would be much easier to use the abstractions provided through fsspec, like gcsfs.

Store now using fsspec:

simonbohnen commented 1 year ago

See #66 for the Boto3Store using s3fs.