microsoft / fluentbit-containerd-cri-o-json-log

Parsing CRI JSON logs with Fluent Bit - applies to fluentbit, kubernetes, containerd and cri-o
MIT License
96 stars 18 forks source link

Logs emitted under `message` instead of `log` #4

Open karlpvoss opened 1 year ago

karlpvoss commented 1 year ago

Bug Report

Description

The provided instructions to create a parser for the cri logs results in the creation of a duplicate parser.

Since 1.0, the fluent-bit docker image has included a built-in list of parsers, one of which is the default cri parser.

Attempting to create another parser with the same name as described here results in an error at start-up:

[error] [parser] parser named 'cri' already exists, skip.

This built-in cri parser emits the main contents of the log under the message field, not the log field, which is contradictory to many of the statements provided.

Expected Behavior

The creation of a parser does not generate errors within fluent-bit. The logs are emitted under the log key, as described.

Reproduce

  1. Take the provided config.yaml and configure a pod running the fluent-bit docker image with a version greater than 1.0.
  2. On start-up the pod should emit a log: [error] [parser] parser named 'cri' already exists, skip.
  3. The logs output by fluent-bit will have the main contents emitted under message instead of `log.

Additional Context

Recommend that the name of the parser specified in this documentation is changed to something like cri-log, as well as the value for Parser under [INPUT].

aolmosj commented 1 year ago

JFYI, the native cri parser stores de raw json app message in the 'message' field, while this repo custom parser stores a cri log format in the 'log' field. Even by using this custom parser, the Merge_Log directive is not working as expected, because the log message is not valid json (it's actually cri with json).

As a workaround I used the native cri parser, deleted the custom parser and added a filter to rename the the field 'message' to 'log'.

[FILTER]
  Name modify
  Match kube.*
  Rename message log