Clarity required around Logging Architecture of Kubelet.

[javadevmtl]

Running Kubernetes 1.27.6

I have some performance issues regarding logrotation where I miss logs, I'm not here blaming Kubernetes, but there needs some clarification around the Log Architecture documents here Loggin Architecture or more details.

We have set containerLogMaxSize and containerLogMaxFiles which works fine and is rotating logs as per the settings. We are trying to debug an issue where we miss logs when there is large volume.

Now here is the disparity.

1. Based on the linked document above and the 2 settings we are lead to believe that Kubelet is doing the rotation, while many online resources, forums, seem to be saying it's logrotate.
2. Looking at the /var/log/pods/ folder indeed the logs are being rotate as per the 2 settings, but they are also gzipped. Nowhere does the document explain this and there's no settings indicated anywhere for compressing the logs. Which leads us to belive that the file is being zipped before the file agent can finish reading all the contents of the file.
3. If logrotate is truly used does Kubelet pass these 2 settings containerLogMaxSize and containerLogMaxFiles to logrotate somehow and if so how (We looked in /etc/logrotate.d/ and /etc/logrotate.conf and found no indication of a Kubenrnetes config pointing to /var/log/pods/)?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten