search

aws / aws-sdk-go-v2

AWS SDK for the Go programming language.
https://aws.github.io/aws-sdk-go-v2/docs/
Apache License 2.0
2.5k stars 602 forks source link

feat: add object io reader/writer to manager #2622

Closed jobstoit closed 2 months ago

jobstoit commented 2 months ago

Hello I wanna contribute an io.Reader for the DownloadManager and io.WriteCloser for the UploadManager. I recently created my own wrapper s3io and figured I could just as well contribute the ObjectReader and ObjectWriter to the s3 sdk. (resolving issue #2247)

Having an io.Reader and io.Write(Close)r can significantly optimise memory consumption in your applications and is therefor a nice to have. I designed it to also work in chunks/parts concurrently similar to how the current downloader/uploader work

This is my first PR here so maybe you can help me with the conventions.

Let me know what you think and if you'd like me to finish this

lucix-aws commented 2 months ago

This is very useful software in its own right but it's not really the interface we want in the transfer manager.

We appreciate the effort but in the interest of saving you time I don't see us accepting any form of contribution to the s3 manager in the near future. In its current state it really needs a holistic redesign. We will most likely pursue releasing a new major version of the feature when that work takes place.

jobstoit commented 2 months ago

Thank you for the response @lucix-aws. Hope the PR could give some inspiration on how to stream s3 objects concurrently. Best of luck with the redesign.

I'll update my s3io package likely too cause I like my implementation I did here better 😅.