fluent-plugins-nursery / fluent-plugin-cloudwatch-logs

CloudWatch Logs Plugin for Fluentd
MIT License
201 stars 141 forks source link

Log event is discarded because it is too large #167

Closed markuman closed 4 years ago

markuman commented 4 years ago

Problem

We handle several tags and multiple cloudwatch log groups with fluentd.

Now we get multipe warnings

fluentd ecs/fluentd/bc852369-4922-426d-a9c4-b2db0e1607c1 2019-11-21 09:34:23 +0100 [warn]: #0 Log event is discarded because it is too large: 357338 bytes exceeds limit of 262144

but we have no idea from which tag this huge message comes from nor to which log group it should goes to.

It would be good to get the group_name in the error message e.g.

https://github.com/fluent-plugins-nursery/fluent-plugin-cloudwatch-logs/blob/48754cac5bf937717bfe5e87147ddd9715a7facd/lib/fluent/plugin/out_cloudwatch_logs.rb#L296

Using Fluentd and CloudWatchLogs plugin versions

markuman commented 4 years ago

Screenshot_2019-11-21 hotpatching error message (6dabe9a1) · Commits · lekker docker fluentd

atm we're patching it like that.