search

Soostone / katip

A structured logging framework for Haskell
203 stars 67 forks source link

Anyone using katip with "loki"? #136

Open bitc opened 2 years ago

bitc commented 2 years ago

Hello, is anyone using katip with the loki log storage system?

loki supports searching through JSON logs: https://grafana.com/docs/loki/latest/logql/log_queries/#json

So I think it should work well, I am just curious if there is anything special that needs to be done, or if anyone has any tips.

Thank you

MichaelXavier commented 2 years ago

I haven't but it looks like promtail is their agent for ingesting logs and sending them to loki. So if promtail understands line-delimited JSON logs, you might just be able to use mkHandleScribeWithFormatter with jsonFormat to format the lines. You could write that to a file and then have promtail ingest it. I'm not sure if it's necessary but you could use logrotate to keep that log file from getting too big.

It also looks like loki has a push api. So you could potentially skip writing files and write a scribe (katip's term for a log exporter) that writes to that API. For that you could use katip-logzio, katip-datadog or katip-elasticsearch as examples. If you end up going that route, consider publishing it to hackage for other loki users.

Hope that helps!