search

reddit / baseplate.py

reddit's python service framework
https://baseplate.readthedocs.io
BSD 3-Clause "New" or "Revised" License
534 stars 172 forks source link

Kombu SQS Compatibility #423

Open isugimpy opened 4 years ago

isugimpy commented 4 years ago

Currently, the connection_from_config method is very AMQP focused. Kombu has underlying support for AWS SQS, but our wrapper to generate connections is unable to take advantage of it because hostname is a required field on the call, but is incompatible with SQS queue URLs in the Kombu SQS implementation. My thought on this is to add an optional arg to connection_from_config to indicate that the queue is SQS (or maybe a list of known strings for different behavior, defaulting to AMQP if the arg isn't specified), and alter the config parser to make hostname optional in that scenario. Would there be another preferred way to implement this?

This isn't fully blocking usage of SQS in baseplate, as engineers can simply create a Connection manually.

spladug commented 4 years ago

I'm looking at Qasim's docs on how he sets up the SQS consumers and it looks like he's passing a hostname of sqs://. Does that pass through the config OK?

isugimpy commented 4 years ago

I'll have to go back and look, but when I last talked to him, he was explicitly creating a Connection object manually and it didn't pass through correctly without that.

spladug commented 3 years ago

@isugimpy did you ever find out if there's more for us to do here? or can this be closed out?