We have been using Medusa for a while to backup our 45TiB Cassandra cluster and since our upgrade from medusa 0.13.4 to 0.20.1, we noticed the upload throughput got a bit lower than usual, now taking twice the time to perform a differential backup.
After some attempts to tweak Medusa, we got some great improvements after updating botocore's max_concurrency, allowing us to reach an upload speed as high as when we were using Medusa 0.13.4.
Before making any contribution: do you think it would be pertinent to include a new configuration entry for the botocore client's max_concurrency setting ?
┆Issue is synchronized with this Jira Story by Unito
┆Issue Number: MED-14
Project board link
Hello!
We have been using Medusa for a while to backup our 45TiB Cassandra cluster and since our upgrade from medusa 0.13.4 to 0.20.1, we noticed the upload throughput got a bit lower than usual, now taking twice the time to perform a differential backup.
After some attempts to tweak Medusa, we got some great improvements after updating botocore's max_concurrency, allowing us to reach an upload speed as high as when we were using Medusa 0.13.4.
Before making any contribution: do you think it would be pertinent to include a new configuration entry for the botocore client's
max_concurrencysetting ?┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: MED-14