search

optiva-media / aws-sqs-xl-messages

The AWS SQS XL Messages Library enables you to manage Amazon SQS message payloads with Amazon S3
MIT License
6 stars 2 forks source link

aws-sdk V3 Support #1

Open MGough opened 3 years ago

MGough commented 3 years ago

Hello 👋🏻 ,

The structure of this library looks great, keeping the existing methods and just injecting the extra functionality allows this to be a nice drop-in solution for existing systems.

Is there any chance you've moved to AWS-SDK V3 internally or will be making the move in this project? It would be great if this library supported it. At a glance it looks like the S3 interface has changed the most. A lot of the SQS API is similar to V2.

cjuega commented 3 years ago

Hi @MGough,

I had a look at aws-sdk v3, they have completely moved to TS which is awesome. They have moved to a class-first approach, so it should be easier to adapt this library (and to actually cover other features it doesn't cover so far).

Unfortunately I can't give you an ETA right now, but we are definitely going to have a look at it!

MGough commented 3 years ago

Thanks for the quick response! The typescript implementation is much nicer. Glad to hear it'll make things simpler.

I might have a go at trying to get a fork working with the new API, if so I'll report back with anything that I encounter.

cjuega commented 3 years ago

I might have a go at trying to get a fork working with the new API, if so I'll report back with anything that I encounter.

Sure, any help would be much appreciated.

I will spend some time during the weekend to have a deeper look at v3.

In a first look, I believe the easiest would be to leverage the new middleware stack feature to rewrite this library:

https://aws.amazon.com/es/blogs/developer/middleware-stack-modular-aws-sdk-js/

MGough commented 3 years ago

I'm not sure I'll be able to provide a useful branch anytime soon as my requirements might slowly drift from the features of this project.

But I have a few things that I've encountered so far that might be useful to you:

  1. S3Client GetObjectCommandOutput is not as nice as in aws-sdk v2, getting it to work in typescript is a bit painful as it is type hinted with some browser only types.
  2. If you're using the middleware approach you'll likely replace the ReceiveMessageCommandOutput.Body from the S3 URI to the S3 Content. The aws-sdk will then attempt to verify the MD5 hash of the Body matches the expected MD5 hash, so you'll need to update it to match the S3 contents, rather than the hash of the S3 URI.