Closed jlathrop closed 5 years ago
Hi @jlathrop,
In general it is very difficult to determine if the loss of logs is caused by a device being removed (loss is the expected) or a failure (loss is an issue) unless you have some way to configure the expected set of devices.
What you can do is to find all devices/sources from which you haven't heard from since the last X minutes/hours.
The following query will list hosts we have not heard anything from within the last 10s. So if you run this query on a live timeinterval that is at least the double of your grace period you will detect when sources disappearing.
groupby(#host, function=max(@timestamp)) | missing:=_max < now() - 10000 | missing=true
Will this work for your use-case?
Regards - @pmech
Closing due to inactivity
Hey, FYI, this triggers if there is no log at all:
The proposed solution above needed at least 1 log
Perhaps this is not something many others would find useful but this is what drove us away from PapertrailApp and to Humio. When a device stops shipping logs for a set period, it would be nice to know that via an alert. So an ability to create an alert for "device" not sending new logs in "xx" time so that we can check into why. With PapertrailApp if a device stops sending logs, they remove it from the interface... That is the only way you would know - but its unknown after how long.
"Alert: Device 'WebHost01' has not shipped any logs in over 72 hours."