awslabs / aws-s3-transfer-manager-rs

Apache License 2.0
5 stars 0 forks source link

Improve Memory Usage #60

Open waahm7 opened 5 days ago

waahm7 commented 5 days ago

We can blow our memory during upload since we don't hook our read_body phase with the scheduler. We need to improve our scheduler so that we acquire a permit before reading the body and maybe release it after uploading, or have different types of permits.

We will need to do something similar for Download as well if user is not pulling parts as fast as possible off the channel.