aws / amazon-cloudwatch-logs-for-fluent-bit

A Fluent Bit output plugin for CloudWatch Logs
Apache License 2.0
170 stars 49 forks source link

[Feature Request] Allow to tag log groups in Helm #317

Closed says-inti closed 1 year ago

says-inti commented 1 year ago

We currently use the helm chart: https://artifacthub.io/packages/helm/aws/aws-for-fluent-bit for setting up fluent bit on aws via cdk.

We would like to be able to tag log groups on creation for the sake of using those tags in other places in code.

It is supported in the code as far as we can see: https://github.com/aws/amazon-cloudwatch-logs-for-fluent-bit/blob/mainline/fluent-bit-cloudwatch.go

With the property: new_log_group_tags.

When applying this we receive: cloudwatch_logs: unknown configuration property 'new_log_group_tags'.

Reproduceable when using this in the configuration yaml:

...
cloudWatch:
  enabled: true
  match: "application.*"
  region: "${AWS_REGION}"
  logGroupName: "/aws/containerinsights/${CLUSTER_NAME}/application"
  logStreamName:
  logStreamPrefix: ${HOST_NAME}-
  logKey:
  logFormat:
  logRetentionDays: 14
  roleArn:
  autoCreateGroup: true
  endpoint:
  credentialsEndpoint: {}
  new_log_group_tags: mytag=tag
additionalOutputs: |
  [OUTPUT]
      Name                cloudwatch_logs
      Match               dataplane.*
      region              ${AWS_REGION}
      log_group_name      /aws/containerinsights/${CLUSTER_NAME}/dataplane
      log_stream_prefix   ${HOST_NAME}-
      auto_create_group   true
      extra_user_agent    container-insights
      new_log_group_tags  mytag=tag
...
says-inti commented 1 year ago

Nevermind. https://github.com/aws/aws-for-fluent-bit/blob/mainline/troubleshooting/debugging.md

It seems it is possible:

The following options are only supported with Name cloudwatch and must be removed if you switch to Name cloudwatch_logs.

default_log_group_name
default_log_stream_name
new_log_group_tags
credentials_endpoint

It was hidden somewhere in documentation :).