Upgrade to boto3

[jbbarth]

Hey @ybastide 👋 I hope everything's good on your side

I'd be VERY interested to upgrade simpleflow to use boto3:

• boto2 has been deprecated for a long time, it doesn't receive much updates anymore, and in particular doesn't support IMDS v2 authentication
• boto3 is rock solid, and I think it won't be a problem for botify except for a few setting bits that will be trivial to upgrade for @dauving and team

Let me know what you think

[ybastide]

👋 Hi! Long time no see 🙂

I'd be delighted, obviously 🎉 Do you plan to do it?

[jbbarth]

Yes I'm happy to do it, I'll take a look in the next weeks and submit a PR. What I'm not 100% clear about is the potential boto.cfg and AWS_* env vars on your infra, but we can deal with that later.

[ybastide]

We're using both, so it will inevitably be OK 🙃

[jbbarth]

@ybastide I started looking into it but things are complicated on the "swf" module querysets:

• the API is sometimes weird (we're forced to repeat the "domain" at times, we compensate some boto2 naming mis-alignments, ...)
• they massively use *args and **kwargs so we can't possibly know how they're used

I'm wondering:

• 1/ if we want to keep this querysets system, or if we should remove it? As far as I know it was not used a lot at Botify, but things may have changed
• 2/ if we keep it, where's the cursor in terms of backwards compatibility breakage we can accept?
  • A/ we want full backwards compatibility (might be a bit hard)
  • B/ best effort ; calling signatures could remain ~the same, but responses might change (in particular boto3 answers with datetime.datetime objects while boto answers with timestamps/integers)
  • C/ we don't care, we can change signatures and responses

I started with "1 = yes, 2 = B", but I'm curious what's your take on this. I'd be fine with "1 = no" or "1 = yes, 2 = C" 😅

[jbbarth]

Nevermind, it's not that hard to go with "1 = yes, 2 = B" in the end. PRs soon.

[jbbarth]

423 is merged, yoohoo!