Fluent bit default log level is set to info, which is pretty chatty. This is causing customers to pay unnecessarily large log bills, particularly when their logs are not in the "kubernetes" format which results in this log message:
If the kubernetes structure is not found in the log record, then the log_group_name and log_stream_prefix will be used instead, and Fluent Bit will log an error like:
...
Other customers have reported a large amount of this unnecessary warning
2023-09-21T20:14:51.860592207Z stderr F [2023/09/21 20:14:51] [ warn] [parser:_ml_cri] invalid time format %Y-%m-%dT%H:%M:%S.%L%z for '2023-09-21T20:14:51.860487043Z stderr F [2023/09/21 20:14:51] [ warn] [parser:_ml_cri] invalid time format %Y-%m-%dT%H:%M:%S.%L%z for '2023-09-21T20:14:51.860372704Z'
Description of changes
Change the default log level in our samples from info to error
License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Tests
I spun up a new cluster and deployed the enhanced CI yaml
I also verified info level application logs are still uploaded to cloudwatch
Requirements
Before committing the code, please verify the following:
If this commit includes changes to existing sample configurations, you acknowledge that you have confirmed this will not impact existing customer behavior.
If not necessary, consider creating a new sample configuration for this change.
…ssary logs being published (#155)" (#156)
This reverts commit 4e4d41ad9501dcce315845606a5ff1e2dd2d8976. and re-applies https://github.com/aws-samples/amazon-cloudwatch-container-insights/pull/155
Description of the issue
Fluent bit default log level is set to info, which is pretty chatty. This is causing customers to pay unnecessarily large log bills, particularly when their logs are not in the "kubernetes" format which results in this log message:
https://docs.fluentbit.io/manual/v/1.9-pre/pipeline/outputs/cloudwatch
Other customers have reported a large amount of this unnecessary warning
Description of changes
Change the default log level in our samples from info to error
License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Tests
I spun up a new cluster and deployed the enhanced CI yaml I also verified info level application logs are still uploaded to cloudwatch
Requirements
Before committing the code, please verify the following: