aws-ia / terraform-aws-eks-blueprints-addons

Terraform module which provisions addons on Amazon EKS clusters
https://aws-ia.github.io/terraform-aws-eks-blueprints-addons/main/
Apache License 2.0
272 stars 127 forks source link

Fluentbit Blueprint regex and timestamp incorrect causes flooding #412

Closed ChandlerKyle closed 2 months ago

ChandlerKyle commented 2 months ago

Description

Versions

Reproduction Code [Required]

 enable_fargate_fluentbit = var.enable_fargate_fluentbit
 fargate_fluentbit = {
   flb_log_cw = true
 }

Steps to reproduce the behavior:

Set flb_log_cw to true as detailed here https://aws-ia.github.io/terraform-aws-eks-blueprints-addons/main/addons/fargate-fluentbit/

Expected behaviour

Logs from fluentbit will flow into cloudwatch

Actual behaviour

Log after log will output in error recursively as it fluentbit fails to match the timestamp from its own internal log generation

Additional context

Current regex from Here ^(?

Also found that the time format would also fail to match Time_Format %Y-%m-%dT%H:%M:%S.%L%z

Example logs

1724223600170,"{""stream"":""06:57:59]"",""logtag"":""["",""message"":""warn] [parser:regex] invalid time format %Y-%m-%dT%H:%M:%S.%L%z for '[2024/08/21'"",""log"":""[2024/08/21 06:57:59] [ warn] [parser:regex] invalid time format %Y-%m-%dT%H:%M:%S.%L%z for '[2024/08/21'""}",fargate-logs-flblogs.var.log.fluent-bit.log
1724223600170,"{""stream"":""06:57:59]"",""logtag"":""[error]"",""message"":""[parser] cannot parse '[2024/08/21'"",""log"":""[2024/08/21 06:57:59] [error] [parser] cannot parse '[2024/08/21'""}",fargate-logs-flblogs.var.log.fluent-bit.log