cAdvisor-less, CRI-full Container and Pod Stats

[haircommander]

Enhancement Description

• One-line enhancement description (can be used as a release note): cAdvisor-less, CRI-full Container and Pod Stats
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2371-cri-pod-container-stats/README.md
• Primary contact (assignee): @haircommander, @bobbypage
• Responsible SIGs: sig-node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.26
  • Beta release target (x.y): 1.28
  • Alpha release target (x.y): 1.29 See https://github.com/kubernetes/enhancements/pull/4249
  • Stable release target (x.y):
• [x] Alpha
  • [x] KEP (k/enhancements) update PR(s):
    • https://github.com/kubernetes/enhancements/pull/2812
    • https://github.com/kubernetes/enhancements/pull/3559
    • https://github.com/kubernetes/enhancements/pull/3439
    • https://github.com/kubernetes/kubernetes/pull/110754
    • https://github.com/kubernetes/kubernetes/pull/113609
  • [x] Code (k/k) update PR(s):
    • 1.23 https://github.com/kubernetes/kubernetes/pull/103095
    • 1.26 https://github.com/kubernetes/kubernetes/pull/110754
  • [x] Docs (k/website) update PR(s): https://github.com/kubernetes/website/pull/30535
• [x] Beta 1.28
  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/3770
  • [x] Code (k/k) update PR(s):
    • https://github.com/kubernetes/kubernetes/pull/118838
    • 1.28 https://github.com/kubernetes/kubernetes/pull/116968
  • [ ] Docs (k/website) update PR(s):
• [ ] Alpha 1.29
  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/4249
  • [ ] Code (k/k) update PR(s):
    • 1.30 https://github.com/kubernetes/kubernetes/pull/122846
  • [ ] Docs (k/website) update PR(s):

[haircommander]

we'll be making a major change to alpha in 1.26

[derekwaynecarr]

/milestone v1.26 /label lead-opted-in

[rhockenbury]

Hello @haircommander and @bobbypage 👋, 1.26 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26 (correct me, if otherwise)

Here's where this enhancement currently stands:

• [X] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: 1.26
• [X] KEP readme has a updated detailed test plan section filled out
• [X] KEP readme has up to date graduation criteria
• [X] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the KEP yaml to reflect the latest milestones.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[rhockenbury]

We would also need to have the milestone stages updated in the KEP yaml - https://github.com/kubernetes/enhancements/pull/3559/files#diff-0ed5ee24cb73f3749f465e3aa48380d7daf115cdb2e1e83bb2c0aa3c4786d662R22

[rhockenbury]

I'll mark this as tracked for enhancements freeze but please update the milestone stages in the KEP yaml.

[haircommander]

It looks to me this was done https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2371-cri-pod-container-stats/kep.yaml#L18-L20

[rhockenbury]

Okay, I think it should be fine. I was referring to this line, but since this went into alpha in v1.23, we should probably leave it as v1.23 rather than updating to v1.26.

Thanks!

[ruheenaansari34]

Hi @haircommander 👋,

Checking in once more as we approach the 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• [x] All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• [x] All PRs are fully merged by the code freeze deadline.

For this enhancement, I could not locate any k/k PRs, other than https://github.com/kubernetes/kubernetes/pull/103095. If you do have k/k PRs open, please link them to this issue.

As always, we are here to help should questions come up. Thanks!

[krol3]

Hello @haircommander 👋, 1.26 Release Docs Lead here. This enhancement is marked as ‘Needs Docs’ for 1.26 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.26 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by November 9. Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Any doubt, reach us! Thank you!

[krol3]

Hello @haircommander and @bobbypage :wave: please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before deadline Tuesday 15th November 2022. Thank you!

[haircommander]

I think we can not have docs for this release. How do we opt-out?

[krol3]

@reylejano , This KEP is keeping the 1.23 version, we don't need to update the doc for 1.26.

[haircommander]

we did have new pieces for alpha retargeted at 1.26, but I don't think they require documentation

[krol3]

Hi @haircommander , I think at least we will need to update the new changes of these feature PodAndContainerStatsFromCRI in https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features

cc: @reylejano

[palnabarun]

https://github.com/kubernetes/website/issues/33509 is relevant here.

[krol3]

@haircommander and @bobbypage could you help with the documentation relevant for the release 1.26?

[haircommander]

I've opened https://github.com/kubernetes/website/pull/37962

[marosset]

/remove-label lead-opted-in /remove-label tracked/yes /label tracked/no /milestone clear

[haircommander]

/milestone 1.27

[k8s-ci-robot]

@haircommander: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to [this](https://github.com/kubernetes/enhancements/issues/2371#issuecomment-1400727311): >/milestone 1.27 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[SergeyKanzhelev]

/milestone 1.27

[k8s-ci-robot]

@SergeyKanzhelev: The provided milestone is not valid for this repository. Milestones in this repository: [v1.24, v1.25, v1.26, v1.27, v1.28]

Use /milestone clear to clear the milestone.

In response to [this](https://github.com/kubernetes/enhancements/issues/2371#issuecomment-1414434743): >/milestone 1.27 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[SergeyKanzhelev]

/milestone v1.27

[bobbypage]

@SergeyKanzhelev can you please add lead-opted-in label?

[SergeyKanzhelev]

/label lead-opted-in

[fsmunoz]

Hello @haircommander 👋, v1.27 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT Thursday 9th February 2023.

This enhancement is targeting for stage beta for 1.27 (please correct me, if otherwise)

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: 1.27
• [X] KEP readme has a updated detailed test plan section filled out
• [X] KEP readme has up to date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would need to update the following:

• Update to the latest KEP template; from what I've seen there is a missing topic in the PRR Scalability about resource exhaustion, there could be more.
• The PRR review is linked with the previous one, once that additional section is covered it will be done.
• Update kep.yaml to reflect current stage information. This will be addressed by #3770

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[haircommander]

thanks for the heads up about template changes! I've updated https://github.com/kubernetes/enhancements/pull/3770 to reflect them

[fsmunoz]

Thank you @haircommander , feel free to ping me when this is merged (I see that there is ongoing discussion) for me to act ASAP. It currently seems to cover everthing.

[marosset]

Unfortunately, this exception hasn't satisfied all of the requirements by the v1.27 code freeze and will be removed from the milestone.

If you feel it is important to include this exception in v1.27 please consider filing an exception as outlined at https://github.com/kubernetes/sig-release/blob/master/releases/EXCEPTIONS.md

[SergeyKanzhelev]

My understanding is that Windows support is being added now. Applying the label to track it:

/milestone v1.28

[SergeyKanzhelev]

/stage beta

[marosset]

My understanding is that Windows support is being added now. Applying the label to track it:

/milestone v1.28

Windows support is being added with https://github.com/kubernetes/kubernetes/pull/116968 /sig windows

[SergeyKanzhelev]

/label lead-opted-in

[Kasambx]

Hello @haircommander 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 1:00 UTC on Friday 16th June 2023..

This enhancement is targeting for stage beta for v 1.28 (correct me, if otherwise)

Here's where this enhancement currently stands:

• [x] KEP readme using the latest template has been merged into the k/enhancements repo.
• [x] KEP status is marked as implementable for latest-milestone: v 1.28
• [x] KEP readme has a updated detailed test plan section filled out
• [x] KEP readme has up to date graduation criteria
• [x] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the following:

• Ensure [PR] (https://github.com/kubernetes/enhancements/pull/3770) is merged.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[haircommander]

I've opened https://github.com/kubernetes/kubernetes/pull/118838 to verify metrics are present as expected

[Rishit-dagli]

Hello @bobbypage @haircommander :wave:, 1.28 Docs Lead here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[Rishit-dagli]

Hey @haircommander , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

[haircommander]

done! https://github.com/kubernetes/website/pull/42012

[Atharva-Shinde]

Hey @haircommander Enhancements team here :wave:

Just checking in as we approach Code freeze at 01:00 UTC Wednesday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

• [X] All the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.
• [ ] All code related PR/s are merged or are in merge-ready state ( i.e they have approved and lgtm labels applied) by the code freeze deadline. This includes any tests related PR/s too.

For this enhancement, it looks like the following code related PR is open and it needs to be merged or should be in merge-ready state before the code freeze commences :

• https://github.com/kubernetes/kubernetes/pull/116968
• https://github.com/kubernetes/kubernetes/pull/118838 (must be merged before Test Freeze i.e 01:00 UTC Wednesday, 26th July, 2023)

This is the code related PR/s that I found on this KEP issue

• https://github.com/kubernetes/kubernetes/pull/116968
• https://github.com/kubernetes/kubernetes/pull/118838

Please keep the issue description up-to-date with all the PR/s that are associated with this KEP and let me know if there are other PR/s in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up. Thanks!

[Atharva-Shinde]

Hello @haircommander 👋, 1.28 Enhancements Lead here.

Unfortunately, the e2e test PR associated with this enhancement was not in the merge-ready state by the code-freeze and hence this enhancement is now being removed from the v1.28 milestone.

If you still wish to progress this enhancement in v1.28, please file an exception request. Thanks!

/milestone clear

[SergeyKanzhelev]

/milestone v1.29

@haircommander can you please send a PR to bump kep.yaml metadata?

[npolshakova]

Hi @haircommander 👋, 1.29 Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage alpha for 1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: 1.29. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• [x] KEP readme has up-to-date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

It looks like https://github.com/kubernetes/enhancements/pull/4249/files will address most of these issues.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[SergeyKanzhelev]

/stage alpha

[npolshakova]

Hi @haircommander, just checking in once more as we approach the 1.29 enhancement freeze deadline this week on 01:00 UTC, Friday, 6th October, 2023. The status of this enhancement is marked as at risk for enhancement freeze.

It looks like when https://github.com/kubernetes/enhancements/pull/4249 is merged in it will address most of the requirements. Let me know if I missed anything. Thanks!

[npolshakova]

With https://github.com/kubernetes/enhancements/pull/4249 merged this enhancement is tracked for enhancement freeze! 🚀

[drewhagen]

Hello @bobbypage @haircommander :wave:, v1.29 Docs Shadow here. Does this enhancement work planned for v1.29 require any new docs or modification to existing docs? If so, please follows the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023. Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

[haircommander]

I don't believe so. thanks!

[a-mccarthy]

Hi @bobbypage @haircommander, 👋 from the v1.29 Release Team-Communications! We would like to check if you have any plans to publish a blog for this KEP regarding new features, removals, and deprecations for this release.

If so, you need to open a PR placeholder in the website repository. The deadline will be on Tuesday 14th November 2023 (after the Docs deadline PR ready for review)

Here is the 1.29 calendar

[AnaMMedina21]

Hello @haircommander @bobbypage 👋, v1.29 Enhancements team here.

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023

Here's where this enhancement currently stands:

• [ ] All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).

• Can we add https://github.com/kubernetes/enhancements/pull/4249 this one to the issue description?

• [x] All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.

With all this, the status of this KEP is tracked for code freeze.

As always, we are here to help if any questions come up. Thanks!

[npolshakova]

Hello @haircommander @bobbypage 👋, 1.29 Enhancements team here!

With all the implementation(code related) PRs merged as per the issue description, this enhancement is now marked as tracked for code freeze for the 1.29 Code Freeze! 🚀

The test freeze is 01:00 UTC Wednesday 15th November 2023 / 18:00 PDT Tuesday 14th November 2023. Please make sure all test PRs are merged in by then. The tracked test PRs for this KEP are:

• https://github.com/kubernetes/kubernetes/pull/118838

Please let me know if there are additional test PRs we should track. Thanks!