josb
commented
2 weeks ago @Swatinem my first instinct is to make it clear to AWS that these are also issues for this project. The compile time issue is definitely on their radar.
A quick search for a Rust S3 client only shows https://github.com/durch/rust-s3 but it looks unmaintained.
We should figure out what we want to do with the AWS SDK going forward.
Right now, it is the single biggest offender related to compile times, and pulls in a huge swath of dependencies. Its so bad I dedicated a blog post to it: https://swatinem.de/blog/bloaty-mcbloat-sdk/ Which itself reminded me about a talk I gave ~4½ years ago about equally bad bloat in their JS SDK: https://viennajs.org/en/meetup/2020-01/optimizing-nodejs-memory-usage
But it is not only about it being horrendously bloated. They also break the API, and especially their error handling all the time. I had to work around one of those issues here:
https://github.com/getsentry/symbolicator/blob/b55fe9b4c221642eacd8772073ebc9be313d6c91/crates/symbolicator-service/src/download/s3.rs#L306-L309
And a recent upgrade has again introduced a ton of fresh errors that were previously handled "correctly".
I’m honestly not sure what to do about this exactly. We do want to use uptodate and maintained crates, however you define "maintained". But the amount of hassle we have with the AWS SDK is in no relation to our usage. We only want to download files, nothing more nothing less, we don’t need the whole kitchen sink.
CC @josb just curious to hear your feedback on this, given you contributed the initial rewrite from
rusototo the official SDK.