search

SumOfUs / sumofus

0 stars 0 forks source link

Alternative to RabbitMQ for Message Handling #2

Closed osahyoun closed 9 years ago

osahyoun commented 9 years ago

RabbitMQ is a very fine messaging system, and I'd typically be excited at the prospect of using it.

But, from experience, hosting and managing such a critical part of a system can be complicated and time consuming (especially when trying to keep downtime to a minimum).

I think we should consider Amazon's managed messaging system.

Amazon's Simple Queue Service

Amazon's SQS is a fully managed messaging system. It's fast, highly available and dirt cheap! We get 1 million requests per month, for free! Thereafter, it's $0.50 per million requests, per month.

Here's a quick diagram to show how SQS could work for us:

untitled drawing 2

You'll notice I've also put our PG database onto AWS. They make it easy to set up, operate, and scale PostgreSQL deployments in the cloud, using their RDS service.

I don't want to sound like an AWS fanboy, but their cloud services are a game changer. They allowed us to do some very cool things, often for just pennies, at the last two startups I worked for.

Let me know what you all think.

EricBoersma commented 9 years ago

Hey, I think this could definitely make sense, and at our use level, I think it would be pretty unlikely we'd end up paying a single penny for the queue service. My question is: do we have a good ruby library for interacting with it? One of the reasons that RabbitMQ made sense was because Bunny was a quality, mature library for interaction. If there's a similar option for AWS, I think this would remove some complexity from the system.

osahyoun commented 9 years ago

Yes, aws-sdk gem wraps their entire API. I'd be happy to write another wrapper (around the wrapper!) for a higher abstraction.

NealJMD commented 9 years ago

I've never managed/admin'd a messaging queue, or to be honest anything cloud-hosted beyond a web server or crawler farm, so I have little to say. However, I've never been burned when I chose to use AWS. I've used S3 a lot and EC2 some, and always had a great experience.

EricBoersma commented 9 years ago

This is now agreed upon as part of the design, so I'm going to close the issue.