Improved multi-numa alignment in Topology Manager

[klueska]

Enhancement Description

• One-line enhancement description (can be used as a release note): An enhancement to the TopologyManager that considers relative distances between NUMA nodes when calculating "best hints" that include more than one NUMA node.

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/3545-improved-multi-numa-alignment/README.md

• Discussion Link: Private slack messages

• Primary contact (assignee): @PiotrProkop

• Responsible SIGs: /sig node

• Enhancement target (which target equals to which milestone):

  • Alpha release target: 1.26
  • Beta release target: 1.28
  • Stable release target: 1.30

• [x] Alpha

  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/3549
  • [x] Code (k/k) update PR(s): https://github.com/kubernetes/kubernetes/pull/112914
  • [x] Docs (k/website) update PR(s): https://github.com/kubernetes/website/pull/37668

• [x] Beta

  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/4079
  • [x] Code (k/k) update PR(s): https://github.com/kubernetes/kubernetes/pull/118816
  • [x] Docs (k/website) update(s): https://github.com/kubernetes/website/pull/41718

[klueska]

/cc @fromanirh /cc @swatisehgal

[ffromani]

thanks for the heads up, I'll review shortly

[derekwaynecarr]

/label lead-opted-in

[Atharva-Shinde]

Hey @klueska 👋, 1.26 Enhancements team here!

Just checking in as we approach Enhancements Freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26

Here's where this enhancement currently stands:

• [X] KEP file using the latest template has been merged into the k/enhancements repo.
• [X] KEP status is marked as implementable
• [X] KEP has an updated detailed test plan section filled out
• [X] KEP has up to date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to: -Get the PR https://github.com/kubernetes/enhancements/pull/3549 merged before Enhancements Freeze to make this enhancement eligible for 1.26 release.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you :)

[Atharva-Shinde]

/milestone v1.26 /label tracked/yes

[k8s-ci-robot]

@Atharva-Shinde: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to [this](https://github.com/kubernetes/enhancements/issues/3545#issuecomment-1261340797): >/milestone v1.26 >/label tracked/yes Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[Atharva-Shinde]

/stage alpha

[Atharva-Shinde]

Hello @klueska 👋, just a quick check-in again, as we approach the 1.26 Enhancements freeze.

Please plan to get the action items mentioned in my comment above done before Enhancements freeze on 18:00 PDT on Thursday 6th October 2022 i.e tomorrow

For note, the current status of the enhancement is marked at-risk :)

[rhockenbury]

We have this marked as tracked for v1.26 with #3549 merged.

[Atharva-Shinde]

Hi @klueska 👋,

Checking in as we approach 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• [X] All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes). If there are any other PRs beyond https://github.com/kubernetes/kubernetes/pull/112914, please include them in the Issue Description too.
• [ ] All PRs are fully merged by the code freeze deadline.

As always, we are here to help should questions come up. Thanks :)

[Atharva-Shinde]

Hey @klueska 👋, just a quick check-in again before 1.26 code freeze at 17:00 PDT Tuesday 8th November 2022 i.e tomorrow. Looks like we would at least need to get the code PR/s: https://github.com/kubernetes/kubernetes/pull/112914 (any more PRs?) merged before the code-freeze :)

[krol3]

Hello @klueska 👋, 1.26 Release Docs Lead here. This enhancement is marked as ‘Needs Docs’ for 1.26 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.26 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by November 9. Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Any doubt, reach us! Thank you!

[rhockenbury]

With https://github.com/kubernetes/kubernetes/pull/112914 merged, we have this marked as tracked for code freeze.

[krol3]

Hello @klueska :wave: please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before deadline Tuesday 15th November 2022. Thank you!

[krol3]

Hi @klueska ! thank you for your doc PR here

[marosset]

/remove-label lead-opted-in /remove-label tracked/yes /label tracked/no /milestone clear

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ffromani]

/remove-lifecycle stale

[SergeyKanzhelev]

@ffromani any plans to work on this in 1.28?

[PiotrProkop]

@SergeyKanzhelev @ffromani @klueska I can/want to help with getting this feature graduate to beta.

[ffromani]

@PiotrProkop if you want to work on the graduation in the 1.28 cycle, I can help with reviews.

[PiotrProkop]

@PiotrProkop if you want to work on the graduation in the 1.28 cycle, I can help with reviews.

Sign me in. Do you have maybe any examples what it takes to graduate feature from alpha to beta?

[ffromani]

@PiotrProkop if you want to work on the graduation in the 1.28 cycle, I can help with reviews.

Sign me in. Do you have maybe any examples what it takes to graduate feature from alpha to beta?

I'll fetch examples. In general the KEP template itself should provide some initial guidelines.

[swatisehgal]

@PiotrProkop if you want to work on the graduation in the 1.28 cycle, I can help with reviews.

Sign me in. Do you have maybe any examples what it takes to graduate feature from alpha to beta?

I'll fetch examples. In general the KEP template itself should provide some initial guidelines.

Here are links of features that we graduated to Beta and might help you:

1. Graduation of PodResource API GetAllocatableResource to Beta in 1.23 release
2. Graduation of CPU Manager Policy Options 1.23 Beta

Also, I can help with reviews as well.

[SergeyKanzhelev]

/milestone v1.28

[SergeyKanzhelev]

/assign @PiotrProkop

[SergeyKanzhelev]

/stage beta /label lead-opted-in

[Atharva-Shinde]

Hello @PiotrProkop @ffromani 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on Thursday, 16th June 2023.

Looks like this enhancement is targeting for stage beta for v1.28

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone:v1.28
• [x] KEP readme has a updated detailed test plan section filled out
• [ ] KEP readme has up to date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would need to update the following:

• Update kep.yaml with latest information.
• Update prod-readiness .yaml file to included beta stage information.
• Add response for this question in the Scalability questionnaire of the KEP readme.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you :)

[Atharva-Shinde]

Hey @PiotrProkop @ffromani, We would just need to update beta stage information inside kep.yaml to 1.28 inside of 1.27 and stable to your expectation. Please make the change within 24 hours else you would have to file an exception request. Thanks :)

[PiotrProkop]

Hey @PiotrProkop @ffromani, We would just need to update beta stage information inside kep.yaml to 1.28 inside of 1.27 and stable to your expectation. Please make the change within 24 hours else you would have to file an exception request. Thanks :)

Thanks for pointing that out. Here is a PR with the fix.

[klueska]

Hi @Atharva-Shinde the PR that @PiotrProkop linked to has now been merged.

[Atharva-Shinde]

Awesome @PiotrProkop @klueska ! With all the KEP requirements in place and merged into k/enhancements the status of this enhancement is now marked as tracked. Please keep the issue description up-to-date with appropriate stages as well.

[katcosgrove]

Hello @PiotrProkop ! 1.28 Docs Shadow here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[PiotrProkop]

Hello @PiotrProkop ! 1.28 Docs Shadow here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

Hey,

Here is the placeholder PR: https://github.com/kubernetes/website/pull/41718

Thanks for the reminder!

[Atharva-Shinde]

Hey again @PiotrProkop @klueska :wave:

Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

• [ ] All the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.
• [ ] All code related PR/s are merged or are in merge-ready state ( i.e they have approved and lgtm labels applied) by the code freeze deadline. This includes any tests related PR/s too.

This is the code related PR/s that I found on this KEP issue

• https://github.com/kubernetes/kubernetes/pull/118816

Please keep the issue description up-to-date with all the PR/s that are associated with this KEP and let me know if there are other PR/s in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up. Thanks!

[ffromani]

@PiotrProkop could you please update by editing the first message https://github.com/kubernetes/enhancements/issues/3545#issue-1383872036 linking the related PRs?

[PiotrProkop]

Hey again @PiotrProkop @klueska 👋

Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

* [ ]  **All** the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.

* [ ]  All code related PR/s are merged or are in merge-ready state ( i.e they have `approved` and `lgtm` labels applied) by the code freeze deadline. This includes any tests related PR/s too.

This is the code related PR/s that I found on this KEP issue

* [topologymanager: Promote support for improved multi-numa alignment in Topology Manager to beta kubernetes#118816](https://github.com/kubernetes/kubernetes/pull/118816)

Please keep the issue description up-to-date with all the PR/s that are associated with this KEP and let me know if there are other PR/s in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up. Thanks!

@Atharva-Shinde @ffromani I don't have edit access to this issue(asked @klueska to update description). Just FYI the implementation PR was merged:

• https://github.com/kubernetes/kubernetes/pull/118816

Documentation PR is opened as WIP I am planning to finish it this week:

• https://github.com/kubernetes/website/pull/41718

[ffromani]

Hey again @PiotrProkop @klueska wave Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 . Here’s the enhancement’s state for the upcoming code freeze:

* [ ]  **All** the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.

* [ ]  All code related PR/s are merged or are in merge-ready state ( i.e they have `approved` and `lgtm` labels applied) by the code freeze deadline. This includes any tests related PR/s too.

This is the code related PR/s that I found on this KEP issue

* [topologymanager: Promote support for improved multi-numa alignment in Topology Manager to beta kubernetes#118816](https://github.com/kubernetes/kubernetes/pull/118816)

Please keep the issue description up-to-date with all the PR/s that are associated with this KEP and let me know if there are other PR/s in k/k we should be tracking for this KEP. As always, we are here to help if any questions come up. Thanks!

@Atharva-Shinde @ffromani I don't have edit access to this issue(asked @klueska to update description). Just FYI the implementation PR was merged:

* [topologymanager: Promote support for improved multi-numa alignment in Topology Manager to beta kubernetes#118816](https://github.com/kubernetes/kubernetes/pull/118816)

Documentation PR is opened as WIP I am planning to finish it this week:

* [topologymanager: document topology manager policy options promotion to beta  website#41718](https://github.com/kubernetes/website/pull/41718)

Hi @PiotrProkop what we usually do in such cases is to create a comment which include (copy-paste) the original checklist adding your items/the new items for GA

[Atharva-Shinde]

@PiotrProkop I've updated the issue description, let me know if there are other PRs that needs to be linked in there.

[PiotrProkop]

@PiotrProkop I've updated the issue description, let me know if there are other PRs that needs to be linked in there.

@Atharva-Shinde Thanks!

[Atharva-Shinde]

Hey @PiotrProkop @klueska 👋 Enhancements Lead here, With https://github.com/kubernetes/kubernetes/pull/118816 merged as per the issue description, this enhancement is now tracked for v1.28 Code Freeze. Thanks!

[npolshakova]

/remove-label lead-opted-in

[vsoch]

hey folks! My team is interested in this KEP - it looks like all the alpha/beta check boxes are purple (merged) and the release contender was 1.28? Was this released with 1.28 and I missed it https://kubernetes.io/blog/2023/08/15/kubernetes-v1-28-release/ or is it still TBA (and possibly not documented) because the issue here is still open? Thanks! And apologies for my naivete about this process.

[PiotrProkop]

@vsoch This is available as alpha option since 1.26 and as beta in 1.28. Check current documentation here: https://kubernetes.io/docs/tasks/administer-cluster/topology-manager/#topology-manager-policy-options

[salehsedghpour]

Hello 👋 1.30 Enhancements Lead here,

I'm closing milestone 1.28 now, If you wish to progress this enhancement in v1.30, please follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board and finally add /milestone v1.30. Thanks!

/milestone clear

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[PiotrProkop]

/remove-lifecycle rotten still discussing how to approach e2e for GA

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[kannon92]

@PiotrProkop what are your plans for this KEP for 1.32?