search

kube-logging / logging-operator

Logging operator for Kubernetes
https://kube-logging.dev
Apache License 2.0
1.51k stars 326 forks source link

Semi structured json log parsing example #1632

Open pepov opened 6 months ago

pepov commented 6 months ago

It would be nice to have an example to demonstrate how to parse semi-structured logs.

Let's consider the following log record:

  "message": [
    "I0104 15:47:24.388872   118 SyslogReporter.cpp:10] APP-METRIC {\"fields\":{\"drop\":0},\"measurement\":\"core.datafield\",\"tags\":{\"host\":\"blahblah-testing-9rtvp\",\"instance\":\"0\",\"source\":\"core.datafield\",\"type\":\"data\",\"worker\":\"0\"},\"time\":\"2024-01-04T15:47:24.000000000Z\"}"
],

We want to be able to parse this log message into a fully structured representation, typically into json. Let's create a test setup with a flow that solves this as a blog post or simply as an example in the docs: https://github.com/kube-logging/logging-operator/blob/4a926ef0ab1c780641cc84c6d3d124c50caea635/pkg/sdk/logging/model/filter/parser.go#L203

Ideally the example should represent both the fluentd and syslog-ng solution.

stale[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!