durch / rust-s3

Rust library for interfacing with S3 API compatible services
MIT License
498 stars 195 forks source link

get_object_stream doesn't stream data #300

Closed andrewbaxter closed 1 year ago

andrewbaxter commented 1 year ago

Describe the bug In sync mode, get_object_stream reads whole file into memory. I'm working on a memory constrainted system and a large file (>1gb) leads to an OOM kill.

To Reproduce

Expected behavior

A reasonable buffer should be used for streaming.

Environment

Additional context

This seems very similar to https://github.com/durch/rust-s3/commit/a4fdbdec508a90e97fcd60deefa037bbd5c1b822

durch commented 1 year ago

Fixed by #301