apache / opendal

Apache OpenDAL: One Layer, All Storage.
https://opendal.apache.org
Apache License 2.0
3.46k stars 486 forks source link

new feature: Implement append object for S3 #5351

Open Xuanwo opened 11 hours ago

Xuanwo commented 11 hours ago

Feature Description

AWS S3 supports append object for directory-buckets now.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-objects-append.html

Problem and Solution

OpenDAL already has the append API, we just need to support append mode for s3.

Users can append to object via:

let writer = op.writer_with(path).append(true).await?;
writer.write(bs).await?;
writer.close().await?;

Additional Context

No response

Are you willing to contribute to the development of this feature?

Xuanwo commented 9 hours ago

NOTE: AWS S3 can append on an existing object created by PutObject or multipart uploads. That's different from other services.

image