search

kubernetes / enhancements

Enhancements tracking repo for Kubernetes
Apache License 2.0
3.42k stars 1.47k forks source link

Add support for a drop-in kubelet configuration directory #3983

Open haircommander opened 1 year ago

haircommander commented 1 year ago

Enhancement Description

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

sohankunkerkar commented 7 months ago

@AnaMMedina21 Thanks for the reminder. Could you exclude the docs PR from the code freeze tracker? The code PR has already merged.

salehsedghpour commented 7 months ago

Hello @haircommander 👋 , Enhancements team here.

With all the implementation(code related) PRs merged as per the issue description:

This enhancement is now marked as tracked for code freeze for the v1.30 Code Freeze!

sreeram-venkitesh commented 5 months ago

Hi @haircommander 👋, 1.31 Enhancements Lead here.

If you wish to progress this enhancement in v1.31, please have the SIG lead opt-in your enhancement by adding the lead-opted-in label and set the milestone to v1.31 before the Production Readiness Review Freeze.

/remove-label lead-opted-in

sreeram-venkitesh commented 5 months ago

CC: @sohankunkerkar

kannon92 commented 1 month ago

Do we have plans to work on this for 1.32?

haircommander commented 1 month ago

I put it in proposed for consideration. It's the earliest it could GA, and I'm not sure there are any end-users yet so we may want to soak it a bit more

haircommander commented 1 month ago

I think we should have more uses before we push for GA. I'm bumping this from the SIG node KEP planning for this release

SergeyKanzhelev commented 1 month ago

I think we should have more uses before we push for GA. I'm bumping this from the SIG node KEP planning for this release

One think I feel we need to change here is the GA criteria: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/3983-drop-in-configuration#ga

Changes I suggest:

I would really like to get feedback (ideally real life feedback) from SIG Windows. I vaguely remember they wanted this feature to drop in some windows-specific changes

ffromani commented 1 month ago 
* Change the default of the argument while in beta with the ability to turn off. We MUST NOT go in GA AND change the logic at that time as the exit criteria suggests. GA must be a no-op for the already enabled beta feature.

+1 (non-binding)

haircommander commented 1 month ago

yeah we could maybe make progress in 1.32 while in beta (defaulting to on) and then in 1.33 push for GA

SergeyKanzhelev commented 1 month ago

This seems to be a piece of feedback: https://github.com/kubernetes/kubernetes/issues/126630

brandond commented 1 month ago

We're looking at using this for K3s and RKE2's managed configuration somewhat soonish. Would be nice if we could get the resolv-conf issue fixed first though.

liggitt commented 1 month ago

The merging process for files in the drop-in dir is incorrect and must be fixed before graduating. That is the root cause of the resolvConf issue (https://github.com/kubernetes/kubernetes/issues/126630#issuecomment-2356125287, https://github.com/kubernetes/kubernetes/issues/79421#issuecomment-2356098145).

Opened https://github.com/kubernetes/kubernetes/pull/127421 with a fix

haircommander commented 1 month ago

/milestone v1.32 /label lead-opted-in

SergeyKanzhelev commented 1 month ago

the PR https://github.com/kubernetes/kubernetes/pull/127421 was merged with two TODO. We need to address todos as a criteria to move forward with this KEP.

liggitt commented 1 month ago

We need to address todos as a criteria to move forward with this KEP.

That's up to you. Those TODOs have to be resolved before kubelet revs its configuration format to v1beta2 or v1, certainly.

SergeyKanzhelev commented 1 month ago

We need to address todos as a criteria to move forward with this KEP.

That's up to you. Those TODOs have to be resolved before kubelet revs its configuration format to v1beta2 or v1, certainly.

Yep. There is little that will help us catch this when we have v1. So I think it's best fixed as part of this KEP.

shecodesmagic commented 3 weeks ago

Hello @haircommander 👋, v1.32 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting for stage stable for v1.32 (correct me, if otherwise)

Here's where this enhancement currently stands:

For this KEP, we would need to update the following:

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

ffromani commented 2 weeks ago

Hello! I'm among the KEP wranglers that sig-node set up to help sig-node KEPs make progress smoothly during the 1.32 timeframe. Hello @haircommander, per https://github.com/kubernetes/enhancements/issues/3983#issuecomment-2341326590 my understanding is we want to make progress in the 1.32 cycle but the GA proper is deferred to 1.33, is my understanding correct? IIUC the next steps are mostly address feedback and address open issue, is my understanding correct? thanks!

sohankunkerkar commented 2 weeks ago

Hello @haircommander, per https://github.com/kubernetes/enhancements/issues/3983#issuecomment-2341326590 my understanding is we want to make progress in the 1.32 cycle but the GA proper is deferred to 1.33, is my understanding correct? IIUC the next steps are mostly address feedback and address open issue, is my understanding correct? thanks!

@ffromani I think we need to update the beta requirements as you rightly pointed out and address TODOs mentioned by Jordan in the recent fix. As I understand it, we will proceed to GA once we have sufficient feedback from users regarding the real-time use cases of this feature.

ffromani commented 2 weeks ago

thanks @sohankunkerkar . The PRR deadline is Oct 3rd, but given your last updates I think we can keep the current status and stay in beta. If you think otherwise, please submit an update to make sure you opt in the PRR process.

johnbelamaric commented 1 week ago

Ok, sounds like we are staying in beta and no need for a PRR - please let me know if I misunderstood, I am taking this off my board.

johnbelamaric commented 1 week ago

I also marked it as beta on the enhancements board, instead of stable.

sohankunkerkar commented 1 week ago

Ok, sounds like we are staying in beta and no need for a PRR

That's right!

hegerdes commented 1 week ago

Don't want to clutter this issue but I had some small problems with the drop-in config dir. One of it got resolved but I don't know if the over has been fixed yet. It is described in https://github.com/kubernetes/kubernetes/issues/124796

TL;DR:
Same config with --config flag and drop-in dir specified at the same time crashed the kubelet with a stacktrace

shecodesmagic commented 1 week ago

Since you have decided to stay in Beta, and with all the requirements fulfilled for Beta this enhancement is now marked as tracked for the upcoming enhancements freeze 🚀

pacoxu commented 1 week ago

As this KEP does not have a Feature gate, it used an env KUBELET_CONFIG_DROPIN_DIR_ALPHA to guard the feature.

https://github.com/kubernetes/enhancements/blob/25777c1bba657eac76eb294c012acbbbd519299c/keps/sig-node/3983-drop-in-configuration/kep.yaml#L26-L27 updated the beta version to 1.31; but Jordan's PR https://github.com/kubernetes/kubernetes/pull/127421 was merged in 1.32. Is this a typo?

kannon92 commented 1 week ago

@pacoxu no, that is not a typo. Jordan merged a bug fix but this was promoted to beta in 1.31.

pacoxu commented 1 week ago

@pacoxu no, that is not a typo. Jordan merged a bug fix but this was promoted to beta in 1.31.

I mean that it should be already beta in v1.30 as the alpha ENV was removed in v1.30.

SergeyKanzhelev commented 1 week ago

yes, it stays in beta for one more release with the idea to pre-enable the drop-in directory and ensure that future version bump of config api will not break this

mbianchidev commented 6 days ago

Hey @haircommander,👋 from the v1.32 Communications Team!

We'd love for you to consider writing a feature blog about your enhancement. Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and it is graduating.

To opt-in, let us know by opening a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we finalize the blog schedule.

hacktivist123 commented 5 days ago

Hello @haircommander 👋 1.32 Docs Shadow here.

Does this enhancement work planned for 1.32 require any new docs or modifications to existing docs? If so, please follow the steps here to open a PR against the dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, October 24th 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

hacktivist123 commented 2 days ago

Hello @haircommander 👋 1.32 Docs Shadow here.

This is just a reminder to open a placeholder PR against dev-1.32 branch in the k/website repo for this (steps available here) for this KEP if it requires new or modifications to existing docs:

The deadline for this is Thursday, Oct 24 at 18:00 PDT. Thanks! 🚀