Closed sunnygoel87 closed 2 years ago
Hi @sunnygoel87,
log_retention_days
should be supported by cloudwatch_logs
plugin: https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch#configuration-parameters. Could I know which aws-for-fluent-bit
version you are using?
Hi @zhonghui12 , Thanks for your response. I used following image to deploy Fluent-bit as DaemonSet on EKS cluster. Please let me know if you need any other details. Thanks.
image: amazon/aws-for-fluent-bit:2.22.0
I see. That's our latest image. Could you please share your config if possible?
@zhonghui12 , I was able to configure the retention policy on LogGroup successfully. After getting the latest container image - I created a custom IAM policy(as shown below) and attached that to worker node InstanceRole. And this setup resolved the issue. Thanks for your support.
Policy Name - CloudWatchAgentPutLogsRetention
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "logs:PutRetentionPolicy", "Resource": "*" } ] }
Hi,
We setup FluentBit agent on Amazon EKS cluster to offload Application logs on Amazon CloudWatch. By default, CloudWatch service retains logs forever. They don't expire. But we want CloudWatch to retain the logs only for 14 days.
Based on that requirement, We added "log_renetion_days" parameter in FluentBit ConfigMap resource and set value to 14 days. After deploying the Deamonset, we see the following error in the POD. Could you please help us resolve this issue ?
[2022/03/03 07:22:00] [error] [config] cloudwatch_logs: unknown configuration property 'log_retention_days'. The following properties are allowed: region, log_group_name, log_stream_name, log_stream_prefix, log_key, extra_user_agent, log_format, role_arn, auto_create_group, endpoint, sts_endpoint, metric_namespace, and metric_dimensions.