aws-samples / amazon-cloudwatch-container-insights

CloudWatch Agent Dockerfile and K8s YAML templates for CloudWatch Container Insights.
MIT No Attribution
162 stars 106 forks source link

fix: add tail parser type to support containerd #133

Open htquach opened 1 year ago

htquach commented 1 year ago

Issue

The containerd runtime generate logs as a non-JSON string. When switched to containerd runtime, fluentd will fail to parse any non-JSON log message and produce a large amount of parse error messages in its container logs.

Here is an open issue at fluentd repo: https://github.com/fluent/fluentd-kubernetes-daemonset/issues/412

docker runtime (a valid JSON string) {"log":"2023-05-02 20:17:16 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0\n","stream":"stdout","time":"2023-05-02T20:17:16.666667387Z"}

containerd runtime (just a string) 2023-05-02T20:17:28.143532061Z stdout F 2023-05-02 20:17:28 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0

Here is an example of a short entry from a fluentd container log.

2023-05-02 19:51:40 +0000 [warn]: #0 [in_tail_fluentd_logs] pattern not matched: "2023-05-02T19:51:17.411234908Z stdout F \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\""

Description of changes:

Add, FLUENT_CONTAINER_TAIL_PARSER_TYPE, parser type to handle the non-JSON messages

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.