twosigma / satellite

Satellite monitors, alerts on, and self-heals your Mesos cluster.
Apache License 2.0
143 stars 18 forks source link

cache-state function broken for riak integration #77

Open cmilloy opened 8 years ago

cmilloy commented 8 years ago

With the changes to specify comets in json files it looks like the comet specification for cache-state needs to be updated for users who store mesos task state in Riak via Satellite.

(defn cache-state
  "Return a map which could be used as input for core/run-test.

   Args:
       threshold: Long, specifies the how many failures would trigger a riemann event with critical state.
       period: Seconds, specifies the period this task / test expected to run.
       conifg: Map, specifies the config of this task / test which is expected to have keys :slave (optional), :riak-url, and :bucket.
   Output:
       Return a map which could be used as input for core/run-test."
  [period config]
    {:command ["echo" "cache-state"]
     :schedule (every period)
     :output (fn [& params]
               (let [{threshold :threshold
                      riak-url :riak-url
                      bucket   :bucket} config
                      failures (try
                                 (cache/cache-state-and-count-failures (:slave-host config/settings) riak-url bucket)
                                 (catch Throwable e
                                   (log/error e "Failed to cache state.")
                                   ;; Return a value strictly larger that the threshold.
                                   (inc threshold)))]
                 [{:ttl (* 5 (.getSeconds period))
                  :service "cache state"
                  :state (if (> failures threshold) "critical" "ok")
                  :metric failures}]))})
cmilloy commented 8 years ago

The comet specification would look like this:

    {"name":"satellite-slave.mesos.recipes/cache-state",
     "period": "5s",
     "params": {"threshold": 5, "riak-url": "http://dns-to-riak-server:8098/riak", "bucket": "bucket-name"}}