reduce default concurrency to something more reasonable

soapiestwaffles / s3-nuke

Nuke all the files and their versions from an S3 Bucket 💣🪣

MIT License

21 stars 1 forks source link

reduce default concurrency to something more reasonable #3

Closed soapiestwaffles closed 4 months ago

soapiestwaffles commented 2 years ago

after testing s3-nuke on a large bucket, I was able to get up to ~8000 objects/sec when I burnt out the AWS rate limit. I figure something more reasonable like 5 concurrency

soapiestwaffles commented 2 years ago

will have to fix #5 first before I can figure out a reasonable amount of concurrency

soapiestwaffles commented 2 years ago

now that #7 is merged, going to run some tests on this

mikelorant commented 4 months ago

Recommend dropping this from 100 to 5. Anything else is going to cause the rate limiter to kick in very quickly.

soapiestwaffles commented 4 months ago

agree!