search

Motiva-AI / clj-sqs-extended

Client library for AWS SQS with support for large messages, up to 2GB. This is a Clojure wrapper for https://github.com/awslabs/amazon-sqs-java-extended-client-lib
MIT License
3 stars 0 forks source link

Feature: Fallback to pure-SQS (without use of S3) somehow #32

Closed Quantisan closed 4 years ago

Quantisan commented 4 years ago

either automatic if messages are small enough or rely on user to dictate which implementation on handle-queue or send-message (whichever makes more sense in terms of seamless implementation)

Quantisan commented 4 years ago

I vaguely recall there's something from the Java library. But I could be wrong.

ghost commented 4 years ago

I'm having a hard time finding this information within the AWS API documentation but by looking at the (documentation within) the source code of the extended library and by closely watching at the output during tests it looks like we don't need to do anything here because the library is already smart enough to check the message size upon sending and only stores it in the bucket "if necessary".

Please refer to:

https://github.com/awslabs/amazon-sqs-java-extended-client-lib/blob/master/src/main/java/com/amazon/sqs/javamessaging/AmazonSQSExtendedClient.java

lines 129ff:

/**
     * <p>
     * Delivers a message to the specified queue and uploads the message payload
     * to Amazon S3 if necessary.
     * </p>

We could force the library to always go with S3 with

.setAlwaysThroughS3

upon client construction if we wanted though. Please advise.

Quantisan commented 4 years ago

oh that's great that by default it's smart enough to keep it simple. could you note this behaviour in the relevant docstring within this library and reference the source?