Is your feature request related to a problem? Please describe.
I observed the following error message because of hitting the max size of gzipped message, and the logs seems dropped as a result.
[2024/08/27 01:07:36] [error] [input:forward:forward.0] gzip uncompress failure
[2024/08/27 01:07:36] [error] [gzip] maximum decompression size is 100MB
Describe the solution you'd like
option1: Remove the check of the max limit of decompression size.
option2: Add a config item to disable to check the max limit of decompression size.
option3: Config item to change the max limit of decompression size
It would be nice to change this check just warning instead of error.
Describe alternatives you've considered
I think typical users don't need this kind of max limit, because modern deployments like k8s has auto recovery features from OOM. Dropping logs is more critical issue for users than OOM.
Additional context
I observed the errors after upgrading fluent-bit from 2.1.10 to 3.1.6, but the validation itself was introduced in 1.8.0.
ksauzz
commented
2 months ago
Is your feature request related to a problem? Please describe.
I observed the following error message because of hitting the max size of gzipped message, and the logs seems dropped as a result.
Describe the solution you'd like
It would be nice to change this check just warning instead of error.
Describe alternatives you've considered
I think typical users don't need this kind of max limit, because modern deployments like k8s has auto recovery features from OOM. Dropping logs is more critical issue for users than OOM.
Additional context
I observed the errors after upgrading fluent-bit from 2.1.10 to 3.1.6, but the validation itself was introduced in 1.8.0.