Metric for inbound log data loss at the collector jira ticket log 1032

[pmoogi-redhat]

Description

Currently in_tail plugin doesn't support publishing of inbound logloss - i.e. difference between total bytes written to disk (logfile) and total bytes collected or read by fluentd. This PR got changes in fluentd/lib/fluent/plugin/in_tail.rb, and fluent-plugin-prometheus/lib/fluent/plugin/in_prometheus_tail_monitor.rb plugins to enable publishing of the below parameters

1. totalbytes_logged_in_disk (written to disk at each unique inode level per container process)
2. totalbytes_collected_from_disk (read or collected by fluentd at each unique inode level per container process)

/cc @alanconway @jcantrill /assign @alanconway

/cherry-pick

Links

• Depending on PR(s):
• Bugzilla:
• Github issue:
• JIRA: https://issues.redhat.com/browse/LOG-1032
• Enhancement proposal: many rotations getting missed by fluentd, next enhancement proposal is to get fluentd know about all actual rotations done by CRIO/conmon process by reading extra meta data such as . Based on what rotations fluentd could track and which one got missed computing log-loss as [no-of-rotations-those-missed * maxsizeoflogfile + sum over all tracked rotations by fluentd as [totalbytes_logged_in_disk - totalbytes_collected_from_disk]

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pmoogi-redhat To complete the pull request process, please assign jcantrill after the PR has been reviewed. You can assign the PR to them by writing /assign @jcantrill in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/openshift/origin-aggregated-logging/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[pmoogi-redhat]

with all the requested changes as per review comments.