Open htquach opened 1 year ago
The containerd runtime generate logs as a non-JSON string. When switched to containerd runtime, fluentd will fail to parse any non-JSON log message and produce a large amount of parse error messages in its container logs.
containerd
fluentd
Here is an open issue at fluentd repo: https://github.com/fluent/fluentd-kubernetes-daemonset/issues/412
docker runtime (a valid JSON string) {"log":"2023-05-02 20:17:16 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0\n","stream":"stdout","time":"2023-05-02T20:17:16.666667387Z"}
{"log":"2023-05-02 20:17:16 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0\n","stream":"stdout","time":"2023-05-02T20:17:16.666667387Z"}
containerd runtime (just a string) 2023-05-02T20:17:28.143532061Z stdout F 2023-05-02 20:17:28 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0
2023-05-02T20:17:28.143532061Z stdout F 2023-05-02 20:17:28 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0
Here is an example of a short entry from a fluentd container log.
2023-05-02 19:51:40 +0000 [warn]: #0 [in_tail_fluentd_logs] pattern not matched: "2023-05-02T19:51:17.411234908Z stdout F \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\""
Add, FLUENT_CONTAINER_TAIL_PARSER_TYPE, parser type to handle the non-JSON messages
FLUENT_CONTAINER_TAIL_PARSER_TYPE
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Issue
The
containerd
runtime generate logs as a non-JSON string. When switched tocontainerd
runtime,fluentd
will fail to parse any non-JSON log message and produce a large amount of parse error messages in its container logs.Here is an open issue at
fluentd
repo: https://github.com/fluent/fluentd-kubernetes-daemonset/issues/412docker runtime (a valid JSON string)
{"log":"2023-05-02 20:17:16 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0\n","stream":"stdout","time":"2023-05-02T20:17:16.666667387Z"}
containerd runtime (just a string)
2023-05-02T20:17:28.143532061Z stdout F 2023-05-02 20:17:28 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0
Here is an example of a short entry from a
fluentd
container log.Description of changes:
Add,
FLUENT_CONTAINER_TAIL_PARSER_TYPE
, parser type to handle the non-JSON messagesLicense
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.