humio / issues

Issue Tracker for Humio
4 stars 2 forks source link

Feature Request: No Log(s) Notification in 'x-period' #51

Closed jlathrop closed 5 years ago

jlathrop commented 5 years ago

Perhaps this is not something many others would find useful but this is what drove us away from PapertrailApp and to Humio. When a device stops shipping logs for a set period, it would be nice to know that via an alert. So an ability to create an alert for "device" not sending new logs in "xx" time so that we can check into why. With PapertrailApp if a device stops sending logs, they remove it from the interface... That is the only way you would know - but its unknown after how long.

"Alert: Device 'WebHost01' has not shipped any logs in over 72 hours."

pmech commented 5 years ago

Hi @jlathrop,

In general it is very difficult to determine if the loss of logs is caused by a device being removed (loss is the expected) or a failure (loss is an issue) unless you have some way to configure the expected set of devices.

What you can do is to find all devices/sources from which you haven't heard from since the last X minutes/hours.

The following query will list hosts we have not heard anything from within the last 10s. So if you run this query on a live timeinterval that is at least the double of your grace period you will detect when sources disappearing.

groupby(#host, function=max(@timestamp)) | missing:=_max < now() - 10000 | missing=true

Will this work for your use-case?

Regards - @pmech

anagrius commented 5 years ago

Closing due to inactivity

tristandostaler commented 2 years ago

Hey, FYI, this triggers if there is no log at all:

type=123 | collect(#type) | missing := concat(["#type"]) == ""

The proposed solution above needed at least 1 log