Metrics for log throttling per file

[mrueg]

Fluentd 1.13 includes a log throttling per file feature. https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#enhancement-2

It would be great if there was a way to publish active throttling through metrics to allow early detection.

[Athishpranav2003]

@ashie can i pick this up if its of actual use in the tail monitor?

[ashie]

We need to modify in_tail to add such log throttling statics:

https://github.com/fluent/fluentd/blob/64595e2ac291a3e1d2d02feefdc58e623b3abb02/lib/fluent/plugin/in_tail.rb#L208-L211

[Athishpranav2003]

Oh I see Fine Can I work on this one?

[Athishpranav2003]

@ashie for this one which metrics would be useful for us? the number of lines read from each file ?

[ashie]

Oh I see Fine Can I work on this one?

Of course :smiley:

@ashie for this one which metrics would be useful for us? the number of lines read from each file ?

Need to consider but counting number of times throttling is occurred might be down-to-earth approach: https://github.com/fluent/fluentd/blob/64595e2ac291a3e1d2d02feefdc58e623b3abb02/lib/fluent/plugin/in_tail.rb#L1177

[Athishpranav2003]

@ashie number of times throttling occured means that how many times we are returning a false whenever the check is failed right? Because when i was taking a look at the code, the count of actual number of lines seems to be not possible to be evaluated since we pause,resume and rotate files.

So i will go ahead and count the number of times we hit the check for limit reached

[Athishpranav2003]

@ashie can we close this issue?

[ashie]

Fixed via #227 Thanks!