buildkite / lifecycled

A daemon for responding to AWS AutoScaling Lifecycle Hooks
MIT License
146 stars 34 forks source link

Usage Example? #10

Closed luckymike closed 6 years ago

luckymike commented 7 years ago

This sounds like a potentially intriguing project, but your docs don't actually explain what it means to "respond" to lifecycle events or what the handler actually does.

How does one actually use this once you get past the AWS setup and lifecycled install?

Thanks, hope this question doesn't seem dumb or flippant, I'm honestly curious. :)

lox commented 7 years ago

Yup, I've done a pretty bad job of explaining this project. I'll update the README!

Basically it's for handling graceful shutdown of EC2 instances, so responding to events is usually calling graceful shutdown scripts of your various daemons. lifecycled handles heartbeating until the shutdown scripts finish.

A "handler" is a script that does something like this:

#!/bin/bash
set -eu
service buildkite-agent stop
while pgrep -u buildkite-agent buildkite-agent > /dev/null; do
  echo "Waiting for all buildkite-agent processes to have stopped..."
  sleep 5
done

A more complicated example might handle the params that are sent to the script that indicate a spot instance termination:

#!/bin/bash
set -eu
echo "Got a $1 event"

if [[ -n "${3:-}" ]] ; then
  echo "Spot instance termination at $3, shutting down immediately"
  killall -QUIT buildkite-agent
  exit 0
fi

# otherwise, stop the agent gracefully  
service buildkite-agent stop
while pgrep -u buildkite-agent buildkite-agent > /dev/null; do
  echo "Waiting for all buildkite-agent processes to have stopped..."
  sleep 5
done

I'd love suggestions on how I could make it more generalised, at the minute it was designed to solve a fairly specific problem.

luckymike commented 7 years ago

How do you specify the handler? Does it need a specific name, or is there a config value for that?

benschwarz commented 6 years ago

Yeah I couldn't spot this either @lox

lifecycled <sqs queue arn> … where does the hook script get specified?

lox commented 6 years ago

I'll update the docs now, sorry folks, I got this working and then moved on without documenting it very well.