Cachet Integration

[jbrooksuk]

Integrate with Cachet to report down time.

[spetz]

Queued as my next task after finishing the MS SQL integration.

[spetz]

So I was looking at the docs, could you point me what should I mostly focus on? I was thinking about creating an integration that would expose the methods for sending HTTP requests to the Cachet API. However I'm not sure which ones are the most important ones and if I should include all of them or is it somehow possible to create a sort of batch/bulk requests?

[jbrooksuk]

Components & Incidents.

Cachet doesn't support bulk requests though, so you'll need to update one thing at a time.

[spetz]

Ok, so basically POST/PUT/DELETE requests for these 2 types, that would be filled with some valuable data like a name of the watcher, description of the performed check etc.?

[jbrooksuk]

Yep :)

[spetz]

Alright, doesn't seem too difficult, hopefully, I'll have some working next week :).

[jbrooksuk]

Awesome! Once it's done, we can do a blog post about the integration! :)

[spetz]

That would be great, I'll let you know as soon as I have something working :).

[jbrooksuk]

👍

[spetz]

Just a quick update - integration is going well so far. And I have one question - it doesn't really matter if I send to the specific endpoint an object that has more properties than it's listed in the docs (so I could reuse them), right?

[jbrooksuk]

Nope, you can send whatever.

[spetz]

Ok, cool, thanks for the confirmation.

[spetz]

I've completed most of the integration and have one remaining piece - set of methods that can accept e.g. iteration or check result object as an argument and send a request to the API. If I understand correctly, the component would be a watcher type (e.g. Web or MongoDB) and the incident would be the result of the monitoring operation, for example the information that something has failed or it works just fine?

[jbrooksuk]

Correct :)

[spetz]

I have a great news, integration is pretty much completed. I've also setup the temporary VM to test the API, you can see the results here http://52.166.244.180 :).

[spetz]

Cachet integration has been officially released! :) And a short info on blog.

[jbrooksuk]

Yay! Thanks!

[spetz]

You're welcome, any feedback will be greatly appreciated :). I had to design some basic flow, so that the same components and incidents wouldn't be created all the time, instead I just update'em based on the name and the actual date.

[janpieterz]

I'm trying to combine both Warden and Cachet and am noticing a couple of things:

• Incidents get created when nothing is wrong. See http://cachet.intreba.nl/, just some examples, but these watchers were never down and still incidents have been created. Wouldn't in be more logical to not create an incident if everything is ok?
• Grouping: as you see I have four web watchers (www. and normal, https:// and http://). These make sure all bindings are correct and every path flows correctly to the website (misconfigured certificates, www binding lost in a deployment to a different machine etc). Currently this results in 4 watchers. I might be missing some config in Warden (have only been playing with it this afternoon) but it would be great if I could group these items group name into one component, which then gets added to a component group in cachet. So for example: website (4 watchers). Database (1 mssql watcher). Storage (1 Azure Table Storage watcher), etc. Maybe this is too much of trying to fit the configurability of Warden into a more straight forward user interface though 😄

[spetz]

As for incidents, the thing is that the Cachet integration has to know in a first place if there were some failures in the past, that's why it creates an incident with the status "Fixed" even when the service being monitored works all the time. I'll need to think how to resolve this issue.

For the grouping, there's a "groupId" parameter, yet it's a global one for the whole integration configuration, so again, this will probably need to be separated so that the watchers could have their own, separate groups.

[janpieterz]

Yeah the incidents is interesting. Maybe it shouldn't create them automatically? I commented it out, if I want to create an incident I'll probably want to make it a manual action but that's of course just me.

The grouping I solved using the same name, and doing the pushing to Cachet a bit more manual. It's a bit ugly, but to be honest I think this is actually good enough, there will be so many different configurations for this that a little bit of config like this should be fine:

var webIterations = iteration.Results.Where(x => x.WatcherCheckResult.WatcherGroup == "ArkeWebsite").ToList();
var webIteration = webIterations.FirstOrDefault(x => !x.IsValid);
if (webIteration == null)
{
    webIteration = webIterations.FirstOrDefault();
}
await cachet.SaveCheckResultAsync(webIteration, true);

var otherIterations = iteration.Results.Where(x => !webIterations.Contains(x));
foreach (IWardenCheckResult checkResult in otherIterations)
{
    await cachet.SaveCheckResultAsync(checkResult, true);
}

[spetz]

Good to know that you've found out a temporary workaround. I think I could set the same "groupId" based on the "WatcherGroup" name (just need an additional configuration for such mapping). As for the incidents I'll need to think about this :).

[spetz]

@janpieterz I've just pushed the fixed code that will hopefully resolve your 2 issues. In order to configure the groups, you can use the WithWatcherGroups() where you provide the mappings between the watcher group name and id used in the Cachet. For the incidents, there's a new parameter "saveValidIncidents" (false by default) that will prevent creating a Fixed incident if there were no errors before. Please let me know if it's good enough.

[janpieterz]

@spetz super, cheers for that, that does make it a lot easier. I also stumbled upon some other interesting things:

• I have multiple component groups in Cachet with the same name (for example my production and staging environment, but also over different products etc). Since the GetComponentAsync method is only mapping to name, I think it should be extended to map to group id too! This way I can have multiple components over multiple groups with similar names. I already did this locally, let me know if you'd like this to be forked.
• I also had to remove the update to Cachet when the status value was equal to the current status value. If I didn't, my subscribers were continuously getting email for each and every update even though no status was changed.

[spetz]

Cool, I've just pushed some changes that might help to resolve your latest issues:

• GetComponentAsync() now makes use of the "name" and "groupId" parameters.
• "updateIfStatusesAreTheSame" and "updateIfStatusIsTheSame" are additional boolean flags (false by default) that will skip updating incidents and/or components if the particular status didn't change.