hiroshi-nakagoe
commented
1 year ago an interesting question, but this "client" library just overrides the boto3.sqs library that simply reads SQS messages. and then, it extendedly grabs messages from SQS, read them, and then retrieves the related S3 objects whose locations are written in the SQS message. this means that this library runs under the condition "when you are explicitly receiving the message".
other than the workaround you said above, the idea that I can come up with is to override boto3.Lambda.create_event_source_mapping to parse the extended SQS message, retrieve S3 objects, and map between the extended SQS messages with the "real" data and the lambda events. to get how to override the function, please refer the procedures like https://github.com/archetype-digital/aws-sqs-extended/blob/11a12329721b96b87bfccc769183f89f5f8bc2c8/aws_sqs_ext_client/extended_messaging.py#L639.
I'm busy for a while, but after the period, I'll try to add such extended functions.
Hi,
this seems to work fine when you are explicitly receiving the message, but when we put a automatic trigger from SQS to lambda, then in that case the message is already inside the event variable of the lambda handler which contains the s3 location of the message not the actually message.
in sqs java library using https://awslabs.github.io/aws-lambda-powertools-java/utilities/sqs_large_message_handling/ we can overide and automatically process the event variable and retrieve the message from the s3.
how can we do the same thing here, do we have the same functionality.
the workaround that I see is we can manually retrieve the message from s3 using boto s3 clinet using the s3 bucket and key.