search

awslabs / amazon-sqs-java-extended-client-lib

An extension to the Amazon SQS client that enables sending and receiving messages up to 2GB via Amazon S3.
Apache License 2.0
213 stars 109 forks source link

MessageS3Pointer should be public #13

Closed mrwanny closed 4 years ago

mrwanny commented 8 years ago

I am trying to retrieve the s3 object url, but the MessageS3Pointer class is not public. Is there any reason why it cannot be public?

manickam91 commented 5 years ago

any update on this? i need to consume this POJO but it is set to non public class

robin-aws commented 5 years ago

Hi there! I'm not particularly inclined to expose what's intended to be internal without good reason. What's your use case for this?

peiche commented 5 years ago

I would also like this class to be public. My use case is this:

Messages are added to a queue through one process. Through the extended client, large messages are put into an S3 bucket. The messages are consumed through a worker environment, which posts each message to itself. I need to deserialize the S3 pointer messages from JSON, but I can't use MessageS3Pointer.class in the objectmapper unless it's changed to public.

robin-aws commented 5 years ago

Hi @peiche - are you able to use the extended client on the worker environment as well? That should be both easier to set up and more robust in the face of future changes to this library.

peiche commented 5 years ago

I can use the extended client on the worker, but I can't use receiveMessage() because it will have already POSTed the message to itself.

robin-aws commented 5 years ago

Got it. So the point is you don't want to read the message content into memory from S3 right after receiving the SQS message but later on.

In this case you might not be getting a lot of benefit from this library anyway, since most of the interesting logic is on the consuming side - the main benefit is transparently handling large payloads without changing the SQS API. You may want to just want to have your producers write to S3 and send the bucket address using your own format. You could even fork MessageS3Pointer.

I might not be that opposed to exposing the class, but this way you won't be blocked on that.

manickam91 commented 5 years ago

The same reason i wanted this class public as well. When i consume a extended payload in my lambda, i am unable to parse the class to find out the s3 object key. Used some work around to extract data from the array. It will be very straight forward if we have this class as public.

Thanks Manickam S

On Tue, 29 Oct, 2019, 22:13 Paul, notifications@github.com wrote:

I would also like this class to be public. My use case is this:

Messages are added to a queue through one process. Through the extended client, large messages are put into an S3 bucket. The messages are consumed through a worker environment, which posts each message to itself. I need to deserialize the S3 pointer messages from JSON, but I can't use MessageS3Pointer.class in the objectmapper unless it's changed to public.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-sqs-java-extended-client-lib/issues/13?email_source=notifications&email_token=ADG7273LNW7IGJATG6AUOKDQRBR3NA5CNFSM4CM62BW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECRHOGY#issuecomment-547518235, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADG7272PZ3XV6VIXMNN4753QRBR3NANCNFSM4CM62BWQ .

adam-aws commented 4 years ago

Hi! The new version 1.1.0 now uses the Payload Offloading Java Common Library For AWS (https://github.com/awslabs/payload-offloading-java-common-lib-for-aws) where this class is now public. It was renamed to PayloadS3Pointer.