search

twosigma / satellite

Satellite monitors, alerts on, and self-heals your Mesos cluster.
Apache License 2.0
143 stars 18 forks source link

Adding new recipes should not require a jar be re-deployed to satellite-slaves #74

Open cmilloy opened 8 years ago

cmilloy commented 8 years ago

Currently there are 3 components to generating a notification for issues raised by satellite:

  1. Grab a metric specified in recipes.clj
  2. Publish a condition if the metric meets some threshold criteria specified in a json file (called a comet I think)
  3. Determine how to handle the condition (notification mechanisms, automated resolution actions, etc.) in riemann-config.clj on the satellite-master.

Unfortunately, adding a new metric (step 1 above) requires that we change the recipes.clj file in the satellite-slave. In order to deploy the change we need to recompile the jar, which is not ideal. It would be nice if the recipes could be stored outside of the jar.