Add CDI devices to device plugin API

[klueska]

Enhancement Description

• One-line enhancement description (can be used as a release note): An enhancement to the DevicePlugin API to include a field for CDI devices.

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/4009-add-cdi-devices-to-device-plugin-api/

• Discussion Link: Private slack messages

• Primary contact (assignee): @klueska, @elezar

• Responsible SIGs: /sig node

• Enhancement target (which target equals to which milestone):

  • Alpha release target: 1.28
  • Beta release target: 1.29
  • Stable release target: 1.31

• [x] Alpha

  • [x] KEP (k/enhancements) update PR(s):
  • https://github.com/kubernetes/enhancements/pull/4011
  • [x] Code (k/k) update PR(s):
  • https://github.com/kubernetes/kubernetes/pull/118254
  • https://github.com/kubernetes/kubernetes/pull/119280
  • [x] Docs (k/website) update PR(s):
  • https://github.com/kubernetes/website/pull/42001

• [x] Beta

  • [x] KEP (k/enhancements) update PR(s):
    • https://github.com/kubernetes/enhancements/pull/4238
  • [x] Code (k/k) update PR(s):
  • https://github.com/kubernetes/kubernetes/pull/121440
  • https://github.com/kubernetes/kubernetes/pull/121254
  • [x] Docs (k/website) update(s):
  • https://github.com/kubernetes/website/pull/43435

• [x] Stable

  • [x] KEP (k/enhancements) update PR(s):
    • https://github.com/kubernetes/enhancements/pull/4487
    • https://github.com/kubernetes/enhancements/pull/4446
    • https://github.com/kubernetes/enhancements/pull/4521
  • [x] Code (k/k) update PR(s):
    • https://github.com/kubernetes/kubernetes/pull/123315
  • [x] Docs (k/website) update(s):
    • https://github.com/kubernetes/website/pull/45144

[SergeyKanzhelev]

/milestone v1.28

[SergeyKanzhelev]

/stage alpha

[SergeyKanzhelev]

/label lead-opted-in

[npolshakova]

Hi @klueska 👋, 1.28 Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC Friday, 16th June 2023.

This enhancement is targeting for stage alpha for 1.28 (correct me, if otherwise.)

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: 1.28
• [ ] KEP readme has a updated detailed test plan section filled out
• [ ] KEP readme has up to date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements.

It looks like https://github.com/kubernetes/enhancements/pull/4011 will address most of these issues

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[npolshakova]

Hi @klueska, checking in once more as we approach the 1.28 enhancement freeze deadline on 01:00 UTC Friday, 16th June 2023. The status of this enhancement is marked as at risk. It looks like https://github.com/kubernetes/enhancements/pull/4011 will address most of the requirements. Let me know if I missed anything. Thanks!

[klueska]

@npolshakova https://github.com/kubernetes/enhancements/pull/4011 has been merged

[Atharva-Shinde]

Hey @klueska With all the KEP requirements in place and merged into k/enhancements, the status of this enhancement is now marked as tracked. Please keep the issue description up-to-date with appropriate stages as well. Thank you :)

[VibhorChinda]

Hello @klueska :wave:, 1.28 Docs Shadow here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[npolshakova]

Hey again @klueska :wave:

Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

• [x] All the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.
• [ ] All code related PR/s are merged or are in merge-ready state ( i.e they have approved and lgtm labels applied) by the code freeze deadline. This includes any tests related PR/s too.

For this enhancement, it looks like the following code related PR/s are open and they need to be merged or should be in merge-ready state before the code freeze commences :

• https://github.com/kubernetes/kubernetes/pull/118254

Also please let me know if there are other PRs in k/k we should be tracking for this KEP. As always, we are here to help if any questions come up. Thanks!

[Rishit-dagli]

Hey @klueska , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

[klueska]

/cc @elezar who is working on this feature

[elezar]

I have created a draft PR: https://github.com/kubernetes/website/pull/42001

[Atharva-Shinde]

Hey @klueska 👋 Enhancements Lead here, With https://github.com/kubernetes/kubernetes/pull/118254 merged as per the issue description, this enhancement is now tracked for v1.28 Code Freeze. Thanks!

[Rishit-dagli]

Hello @klueska @elezar wave: please take a look at Documenting for a release - PR Ready for Review to get your docs PR ready for review before Tuesday 25th July 2023. Thank you!

Ref: https://github.com/kubernetes/website/pull/42001

[npolshakova]

/remove-label lead-opted-in

[klueska]

I have added a KEP update link to this issue, to transition this feature to beta in 1.29

[npolshakova]

Hello @klueska, 1.29 Enhancements team here! Is this enhancement targeting 1.29? If it is, can you follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board? Thanks!

[bart0sh]

@mrunalp @SergeyKanzhelev Can you add lead-opted-in label to this issue, please? The issue was included into the SIG-Node 1.29 planning document long time ago.

[bart0sh]

@mrunalp @SergeyKanzhelev can you also change milestone to 1.29 ?

[SergeyKanzhelev]

/milestone v1.29

[SergeyKanzhelev]

/label lead-opted-in

[sanchita-07]

Hello @klueska 👋, 1.29 Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage beta for 1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: 1.29.
• [x] KEP readme has up-to-date graduation criteria
• [x] KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP, it looks like merging of #4238 will be good to go for marking the KEP as tracked for enhancement freeze.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well.

Thank you!

[sanchita-07]

With all the KEP requirements in place and merged into k/enhancements, this enhancement is all good for the upcoming enhancements freeze 🚀. The status of this enhancement is marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank You!

[Princesso]

Hey there @klueska ! 👋, v1.29 Docs team shadow here. Does this enhancement work planned for v1.29 require any new docs or modifications to existing docs? If so, please follow the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023. Also, take a look at Documenting for a release to get yourself familiarized with the docs requirement for the release. Thank you!

[bart0sh]

@Princesso done: https://github.com/kubernetes/website/pull/43435

[sanchita-07]

Hey again @klueska 👋, 1.29 Enhancements team here,

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023:

Here's where this enhancement currently stands:

• [ ] All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• [ ] All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

Please update the Issue description to include all the related PRs of this KEP. The status for this KEP is currently at risk for code freeze.

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up ✌. Thanks :)

[elezar]

@klueska the code changes for this are in https://github.com/kubernetes/kubernetes/pull/121440 could you update the description?

[kcmartin]

Hi @klueska , 👋 from the v1.29 Release Team-Communications! We would like to check if you have any plans to publish a blog for this KEP regarding new features, removals, and deprecations for this release.

If so, you need to open a PR placeholder in the website repository. The deadline will be on Tuesday 14th November 2023 (after the Docs deadline PR ready for review)

Here is the 1.29 calendar

[sanchita-07]

Hello @klueska 👋, 1.29 Enhancements team here.

Making sure are the following PRs related to the current KEP? With all the following implementation(code related) PRs merged into k/k, this enhancement is now marked as tracked for code freeze for the 1.29 Code Freeze! 🚀 Are there any other PRs other than the following ones? If there are any other PRs that need to be tracked, please let us know. Also please update the issue description with the following PR:

• https://github.com/kubernetes/kubernetes/pull/121254

The test freeze is 01:00 UTC Wednesday 15th November 2023 / 18:00 PDT Tuesday 14th November 2023. Please make sure all test PRs are merged in by then. The tracked test PR for this KEP is:

• https://github.com/kubernetes/kubernetes/pull/121440

Please let me know if there are additional test PRs we should track. Thanks!

[bart0sh]

@sanchita-07 Your PR list looks correct. Both PRs have been already merged. There is also one documentation PR. It's pending approval atm: https://github.com/kubernetes/website/pull/43435

[salehsedghpour]

/remove-label lead-opted-in

[salehsedghpour]

Hello 👋 1.30 Enhancements Lead here,

I'm closing milestone 1.29 now, If you wish to progress this enhancement in v1.30, please follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board and finally add /milestone v1.30. Thanks!

/milestone clear

[bart0sh]

@mrunalp @SergeyKanzhelev We're going to promote this to GA in 1.30. Can you change milestone to 1.30 and label this issue with lead-opted-in?

[aojea]

@mrunalp @SergeyKanzhelev We're going to promote this to GA in 1.30. Can you change milestone to 1.30 and label this issue with lead-opted-in?

@bart0sh please ensure then that this CRI fields does not depend in an alpha feature https://github.com/kubernetes/enhancements/pull/3731, as mentioning that CDI CRI fields are independent on DRA , You can not make something GA that depends on another feature that is not GA, because if something happens and those fields change or are removed we'll have a problem

[bart0sh]

@aojea It's the other way around. DRA depends on the CDIDevice CRI field, at least to my understanding. We can't easily change that field as this feature (CDI devices support in Device Plugins) depends on it and it's already graduated to Beta in 1.29.

[aojea]

@aojea It's the other way around. DRA depends on the CDIDevice CRI field, at least to my understanding. We can't easily change that field as this feature (CDI devices support in Device Plugins) depends on it and it's already graduated to Beta in 1.29.

I see, I didn't have it clear so that is why I asked, I'm completely +1 on this, learned about CDI from @klueska the other day and seems a really nice additon

[SergeyKanzhelev]

/stage stable /milestone v1.30

[salehsedghpour]

Hello @bart0sh, 1.30 Enhancements team here! Is this enhancement targeting 1.30? If it is, can you follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board? Thanks!

[bart0sh]

Hello @bart0sh, 1.30 Enhancements team here! Is this enhancement targeting 1.30? If it is, can you follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board? Thanks!

@SergeyKanzhelev @mrunalp Can you add lead-opted-in label to this issue please?

[mrunalp]

/label lead-opted-in

[meganwolf0]

Hello @klueska, @elezar 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 9th February 2024.

This enhancement is targeting for stage stable for 1.30 (correct me, if otherwise)

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: v1.30. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• [X] KEP readme has up-to-date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP, we would just need to update the following:

• KEP Readme missing a Proposal section
• Update the kep.yaml for stage and latest-milestone
• Update the PRR with stable

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[bart0sh]

@meganwolf0 Thank you for the review!

For this KEP, we would just need to update the following:

- KEP Readme missing a Proposal section
- Update the kep.yaml for stage and latest-milestone
- Update the PRR with stable

https://github.com/kubernetes/enhancements/pull/4446 updates kep.yaml and PRR as suggested. @klueska @elezar Can you help to fill Proposal section?

[elezar]

Thanks @bart0sh. I have created #4487

[bart0sh]

@meganwolf0 All requested changes should be in the KEP when #4487 and #4446 are merged.

[bart0sh]

@meganwolf0 both PRs got merged, please review the KEP again, thanks!

[meganwolf0]

With all the requirements fulfilled this enhancement is now marked as tracked for the upcoming enhancements freeze 🚀

[celestehorgan]

Hi @klueska @elezar!

Reminder that this needs a docs PR by the end of the week for the 1.30 release! :)

Even if there's no other updates to functionality as this is going stable, please remember to update your feature flags to stable ✨

[Checksumz]

Hi @klueska ,

👋 from the v1.30 Communications Team! We'd love for you to opt in to write a feature blog about your enhancement!

We encourage blogs for features including, but not limited to: breaking changes, features and changes important to our users, and features that have been in progress for a long time and are graduating.

To opt in, you need to open a Feature Blog placeholder PR against the website repository. The placeholder PR deadline is 27th February, 2024. Here's the 1.30 Release Calendar

[bart0sh]

@celestehorgan

Reminder that this needs a docs PR by the end of the week for the 1.30 release! :)

Thank you for the reminder. Submitted documentation update: Device Plugins: add info about GA graduation

[meganwolf0]

Hey again @klueska, @elezar👋 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Wednesday 6th March 2024 .

Here's where this enhancement currently stands:

• [ ] All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• [ ] All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, I couldn't find any (open) PRs in k/k. With this, it is now marked as at risk for code freeze for the v1.30 Code Freeze!

Please let me know if there are other PRs in k/k we should be tracking for this KEP. As always, we are here to help if any questions come up. Thanks!