search

slatedb / slatedb-go

A cloud native embedded storage engine built on object storage.
https://slatedb.io
Apache License 2.0
51 stars 8 forks source link

Conditional Writes to object store #9

Open naveen246 opened 1 month ago

naveen246 commented 1 month ago

We are currently using thanos-io/objstore lib to write to object store. Right now we have a putIfNotExists() method which is not atomic. https://github.com/slatedb/slatedb-go/blob/v0.1.3/slatedb/object_store.go#L39

The thanos-io/objstore library does not yet have an API that supports conditional writes. https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3-conditional-writes/

They have an open issue for it. https://github.com/thanos-io/objstore/issues/129

See if we wait for them to implement or explore using other libraries ? https://github.com/minio/minio-go https://github.com/apache/opendal/tree/main/bindings/go https://github.com/apache/opendal/issues/5097

Relevant discussion: https://github.com/slatedb/slatedb/issues/45

Xuanwo commented 1 week ago

Conditional Writes are now supported within opendal, and we are eager to extend this support to the OpenDAL Go binding. Please let me know if you need any assistance in this area.