search

elifesciences / elife-bot

tools for creating an automatic publishing workflow.
MIT License
19 stars 10 forks source link

Which service should we use to spin up our instances for running the worker services #1

Closed IanMulvany closed 11 years ago

IanMulvany commented 11 years ago

I just had a look at the OpsWorks introductory video. I think that http://aws.amazon.com/cloudformation/ might be a better option for spinning up instances.

IanMulvany commented 11 years ago

Here is some documentation on integrating this with Chef. https://s3.amazonaws.com/cloudformation-examples/IntegratingAWSCloudFormationWithOpscodeChef.pdf

IanMulvany commented 11 years ago

I think in addition we could set up a reserve micro instance running full time, that could act as the water on the s3 volume, set it to check the volume a couple of times per day, and then spin up the SWF on identification of an event coming in, see pricing here: http://aws.amazon.com/ec2/pricing/#reserved

gnott commented 11 years ago

Full agreement on the micro instance 24/7. It can provide the heartbeat monitoring and run at least one SWF decider instance. Any workflows started by it (or elsewhere) can then be processed.

IanMulvany commented 11 years ago

In terms of the micro instance, our elife-aritlces bucket is openly readable, and if you look at http://s3.amazonaws.com/elife-articles you will see a modification time and etag for each item in the bucket. I'd say we just need to ping that a couple of times a day.

IanMulvany commented 11 years ago

With that in mind, I think I can just go ahead and set up a reserve instance for us to play with. Shold I wait until we have the auto configuration worked out first? That might make more sense.

gnott commented 11 years ago

Good idea on the bucket monitor and the modified times. I say go for it and set up the reserved instance. We can tweak as we go (keeping notes) and document that later into an autoconfig / recipe.