WebAssembly / wasi-blobstore

13 stars 11 forks source link

Intended use when working with multiple blob stores? #27

Open itowlson opened 1 week ago

itowlson commented 1 week ago

In wasi-keyvalue, a store is a first class citizen: a consumer of the API can open multiple KV stores and read and write each of them.

In wasi-blobstore, a consumer can open and work with multiple containers; as with KV stores, the locations and backing stores are abstracted away behind a container-name. So an implementation could allow different containers to be mapped onto different stores, e.g. the cat-pictures container could be mapped to Azure Blob Storage, and the dog-pictures container to AWS S3. Essentially get-container functions as the equivalent of Store::open from the KV world.

However, it's not clear to me how create-container and delete-container are intended to work in this model. As these take a container-name, their implementation is obvious if we back naively onto a single store (e.g. "containers map 1-1 onto Azure containers in such-and-such account"), but if WASI containers are mapped across multiple backing stores, how does the consumer specify which store is intended?

Or is this presumed to be down to host policy (e.g. "all container names starting with cat are created on Azure Premium Storage, all container names starting with dog are created on /dev/null")?

Or is it intended that consumers import a separate instance of the blobstore interface for each concrete backing store?

Sorry if this is in the docs - I did have a look but didn't see anything.

Mossaka commented 22 hours ago

You've raised a great point. I do believe that the APIs of wasi-keyvalue are designed to avoid the issues about how to manage the store (create / delete a store). Perhaps we should rethink if create-container is really needed? If it's needed, I would suggest that we scope it to single blob store operations.

CC @thomastaylor312