HubSpot / Singularity

Scheduler (HTTP API and webapp) for running Mesos tasks—long running processes, one-off tasks, and scheduled jobs. #hubspot-open-source
http://getsingularity.com/
Apache License 2.0
823 stars 188 forks source link

S3Uploader aws client usage efficiency #2198

Closed ssalinas closed 3 years ago

ssalinas commented 3 years ago

Previously we spun up one client per S3Uploader object, since they often have separate creds. However, many are shared, and duplicates become a problem when we start hitting thousands of uploader objects.

Open questions:

cc @baconmania

baconmania commented 3 years ago

🚢