openshift-ci-robot
commented
3 years ago [APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: alanconway, pmoogi-redhat
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Needs approval from an approver in each of these files:
- ~~[OWNERS](https://github.com/openshift/origin-aggregated-logging/blob/master/OWNERS)~~ [alanconway]
Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
openshift-bot
Description
Currently plugin implementations - fluentd/lib/fluent/plugin/in_tail.rb, and fluent-plugin-prometheus/lib/fluent/plugin/in_prometheus_tail_monitor.rb don't support publishing of inbound logloss - i.e. difference between total bytes written to disk (logfile) and total bytes collected or read by fluentd. This PR proposes a new fluent plugin to publish total bytes collected to disk. This is done to avoid any changes required in in_tail.rb and fluent-plugin-prometheus/lib/fluent/plugin/in_prometheus_tail_monitor.rb. Note total bytes logged will get published by a separate go file watcher PR: https://github.com/openshift/cluster-logging-operator/pull/979
log_collected_bytes_total (read or collected by fluentd per container process)
/cc @alanconway @jcantrill /assign @alanconway
/cherry-pick
Links
Depending on PR(s):
Bugzilla:
Github issue:
JIRA: https://issues.redhat.com/browse/LOG-1213
Enhancement proposal: many rotations getting missed by fluentd, next enhancement proposal is to get fluentd know about all actual rotations done by CRIO/conmon process by reading extra meta data such as . Based on what rotations fluentd could track and which one got missed computing log-loss as [no-of-rotations-those-missed * maxsizeoflogfile + sum over all tracked rotations by fluentd as [totalbytes_logged_in_disk - totalbytes_collected_from_disk]