Open calebhailey opened 3 years ago
What if we enabled an SQL interface to accomplish this? We could stream the log into an in-memory sqlite database and allow users to produce metrics with SQL.
For instance, assume our log is a series of JSON objects, and we make the top level keys available:
{"verb": "GET", "status": 200, "user": "kermit"}
{"verb": "POST", "status": 301, "user": "ms. piggy"}
{"verb": "GET", "status": 404, "user": "cookie monster"}
WITH ok_status(count) AS (SELECT count(status) FROM log WHERE log.status = 200)
WITH kermit_requests(count) AS (SELECT count(user) FROM log WHERE log.user = 'kermit')
SELECT ok_status.count AS ok_requests, kermit_requests.count AS kermit_requests
FROM ok_status, kermit_requests;
sensu-check-log
can use the resulting table to create Sensu metrics. We could even get fancy and output labels.
Add support for extracting actual metrics reported in a log file, or generating/calculating metrics based on some trend or "pattern" in a log file (e.g. "N errors in the last 10 seconds"). The latter capability more or less already exists via the
-match
flag, but it would need to produceevent.metrics
as a result.