search

dogsheep / dogsheep-photos

Upload your photos to S3 and import metadata about them into a SQLite database
Apache License 2.0
170 stars 15 forks source link

Try running S3 uploads in a thread pool #11

Closed simonw closed 4 years ago

simonw commented 4 years ago

Since #10 provided such a speedup, can the same thing be done for the actual uploads?

http://ls.pwd.io/2013/06/parallel-s3-uploads-using-boto-and-threads-in-python/ suggests it can really help performance.