search

mso-net / lucee-aws

Lucee extension to provide simpler access to common AWS commands through the AWS SDK
MIT License
25 stars 6 forks source link

Add a basic SQS integration #26

Open SimonHooker opened 9 years ago

SimonHooker commented 9 years ago

I can see more of a use case for this than SNS but still this requires a bit of investigation to figure out

timmaybrown commented 6 years ago

Is this project still being developed? I'm needing to implement SQS. Was going to look into what it will take to work with SQS and ACF/lucee, but wasn't sure if any work on this front had already begun.

Thanks!

SimonHooker commented 6 years ago

Hi Tim,

Lately it's not really been worked on too much because the functionality created filled the need that we had. To be honest lately we've mostly used Node scripts and the Javascript SDK in place of developing further functionality within lucee-aws as we'd like to be able to ultimately remove the AWS-SDK jars due to just how large they are.

Depending exactly what functionlity you were looking for thuogh, I might be able to whip something up within this project easily enough :) What were you hoping to achieve?

timmaybrown commented 6 years ago

I guess I'm just trying to, better understand how to use queue base processing within my application for distributed processes and SQS seemed to a natural piece of that puzzle since we were already using SNS to notify some HTTP endpoints. SNS has some re-try settings, but SQS seemed to be more durable in the event that the server wasn't available for whatever reason.

Just trying to figure out how to consume an SQS queue from coldfusion. Might look into using the amazon-sqs-java-messaging-lib.jar to consume the queue through an event gateway since it's just an interface for JMS it appears (all things I'll have to research a lot more to fully understand).

I think I might be blurring the lines here since this is Lucee extension, so I apologize if I'm off base.

Thanks!

SimonHooker commented 6 years ago

I did start looking at SQS for a similar reason but in the end put together my own simple MySQL-backed queue which was fit for my purpose. I believe at the time FIFO queues weren't quite available yet and processing the same message multiple times would have been detrimental. Regarding availability, the MySQL database is a multi-AZ RDS so about as reliable as we can easily get.

I'm afraid I've never really touched on event gateways. When we got to looking at that kind of functionality we ended up diverting into Lambda, and from there into our own Lambda-esque cluster to work around some of the restrictions and insecurity within Lamdba.

I'm not sure of your exact use case and it probably differs massively from my own. When I raised this it was with a view to retiring Bamboo being executed internally to run generated task queues. The intention was that we would queue up the chain of tasks for each client. The lack of FIFO support at the time would have potentially resulted in queues multiplying / extra handling being required on our end to ensure each message only got processed exactly once thus resulting in similar logic to I had to ultimately implement in my homemade message queue.

Adding in SQS to here is probably not a big deal. The request/responses seem to just be simple in/out operations so it just comes down to what methods you'd need to use. Unfortunately because my experience with SQS is limited to nil, developing something useful would be pretty difficult without further information especially when it comes to ensuring there are suitable unit tests around it.