search

poundifdef / smoothmq

An improved drop-in replacement for SQS
https://www.smoothmq.com
GNU Affero General Public License v3.0
2.04k stars 35 forks source link

config: Add `--sqs-endpoint` option #24

Closed sundbry closed 1 month ago

sundbry commented 1 month ago

Hi @poundifdef, thank you very much for sharing this program. I'm very excited to have this simple tool in my kit.

I was writing some test cases for a Clojure client using amazonica which is sugar around the underlying official AWS SDK, and ran into a roadblock sending messages to a queue. Due to the queue URLs being advertised as https://sqs.us-east-1.amazonaws.com/<tenant>/<queue-name>, the official Java SDK was failing to send messages since it was trying to hit the actual amazonaws.com endpoints.

Here is a simple patch to publish true queue URLs. I kept the default behavior here to pretend to be sqs.us-east-1.amazonaws.com for compatibility.

Provide an option to set the advertised endpoint for the SQS queue URLs. The official AWS client for Java will connect to the queue url as advertised by the server, regardless of what the endpoint is configured to in the top level AWS client.

When running SmoothMQ behind a load balancer, this should be the public endpoint of the server.

poundifdef commented 1 month ago

This is great, thank you. Curious: can you tell me more about how you're using it? (Is it for dev environments, testing, prod applications, or something else?)

sundbry commented 1 month ago

I'm using it to build jupitercloud.com right now,. I would be thrilled to list/integrate your SaaS edition in the catalog BTW if you had an API I could use as a reseller to create tenants.

On a technical note, I'm looking for a solution that can create lots of micro queues (provided, some of them may grow very large), to replace an architecture where I am using a single Pulsar topic in my backend. Basically one queue per vendor on the platform to make scheduling of background work more even.