Closed albertalexandrov closed 1 month ago
Overall, the S3 upload_fileobj implementations are not equal, they're similar but not equal, as boto3 calls out to the s3transfer module, and here we attempt to replicate it as best as we can. That being said, I'd expect it to take less than a second, so that seems a bit odd.
Try caching and reusing the async client, there's no need to make a client on every request, same applies to the session. Look into setting up an AsyncExitStack, (that or just calling session.client().__aenter__()
, though thats slightly less clean)
I create a session once at the start of application and use it. It helped.
Hi there!
I need to upload multiple files to S3 in parallel. The script I use to upload:
And there is an attachment service I upload to. If I upload files like that:
then I get next results:
As you could notice it's rather slow for uploading 20 bytes.
Then if I do not create session each time but create it once:
It becomes much faster:
What's wrong with first case?