How to work with structured logs?

SumoLogic / sumologic-collector-docker

A Sumo Logic collector for Docker.

Apache License 2.0

69 stars 55 forks source link

How to work with structured logs? #93

Open diegobernardes opened 4 years ago

diegobernardes commented 4 years ago

I'm a SumoLogic user and my app is generating structured JSON logs. Using this collector the logs are ingested as plain text lines. There is any way to enable JSON processing?

maimaisie commented 4 years ago

I believe the Docker API we are using for our Docker sources emits logs line by line, and currently Sumo does not support stitching log lines back once we receive the data from Docker.

An alternative is to use file based Docker logging driver for your containers and configure a Local File source in the collector to ingest data from it. This way, you can take advantage of automatic boundary detection or specify your own boundary detection rule with regex for multiline messages.

himanshub16 commented 4 years ago

https://github.com/moby/moby/issues/22920#issuecomment-298153011

Agreed. We decided to make all our services log in json format and it works like a charm.