Deprecate and remove kustomize from kubectl

[soltysh]

Enhancement Description

• One-line enhancement description (can be used as a release note): Deprecate and remove kustomize from kubectl
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/4706-deprecate-and-remove-kustomize/README.md
• Discussion Link: https://docs.google.com/document/d/1I1UFGHMDO7mMbDbioQp52DEJXEhk1qymch3qL5-EN10/edit#bookmark=id.da6ykky890fn
• Primary contact (assignee): @soltysh
• Responsible SIGs: sig-cli
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.31
  • Beta release target (x.y):
  • Stable release target (x.y):
• [ ] Alpha
  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/4712
  • [ ] Code (k/k) update PR(s):
  • [ ] Docs (k/website) update PR(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[soltysh]

/sig cli /milestone v1.31 /stage alpha /label lead-opted-in

[dipesh-rawat]

Hello @soltysh 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on on 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024.

This enhancement is targeting for stage alpha for 1.31 (correct me, if otherwise).

Here's where this enhancement currently stands:

• [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
• [ ] KEP status is marked as implementable for latest-milestone: { CURRENT_RELEASE }. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• [ ] KEP readme has up-to-date graduation criteria
• [ ] KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline so that the PRR team has enough time to review your KEP.

For this KEP, we would need to update the following:

• Create the KEP readme using the latest template and merge it in the k/enhancements repo.
• Ensure that the KEP has undergone a production readiness review and has been merged into k/enhancements.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[dipesh-rawat]

Hello @soltysh 👋, 1.31 Enhancements team here.

Now that PR https://github.com/kubernetes/enhancements/pull/4712 has been merged, all the KEP requirements are in place and merged into k/enhancements, this enhancement is all good for the upcoming enhancements freeze. 🚀

The status of this enhancement is marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[Princesso]

Hello @soltysh :wave:, 1.31 Docs Lead here. Does this enhancement work planned for 1.31 require any new docs or modification to existing docs? If so, please follow the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT. Also, take a look at Documenting for a release to get yourself familiarised with the docs requirement for the release. Thank you!

[soltysh]

Hey @Princesso we'll probably want to put together a blog post around 1.31 release, to more advertise the fact of this deprecation along with the future plan for removal. So that more users are aware of this fact. I'll followup with appropriate PRs.

[liggitt]

I'm surprised and disappointed to see this proposed. I don't love the current state, but it's the commitment we made to users... we should not just break them without very very good reasons.

I think the KEP enormously underestimates the impact. There are thousands of publicly visible uses and likely even more non-public uses. Dropping support / breaking those uses does reputational damage to kubernetes for being unstable in new versions.

The justification / motivation in the KEP is vague

• "risking users of kubectl to work with outdated version of kustomize" isn't an issue for users whose existing manifests are working properly
• "some of the kustomize dependencies has already been problematic to the core kubernetes project" needs more details ... I wasn't aware of ongoing problematic dependencies here

[thockin]

Without having read the KEP, just the headline....I'm pretty strongly in the "no, that would break users" camp.

I'm all for throwing warnings - use colors and flashing terminal codes, heck - make it play the Star Wars alarm siren on the PC speaker if you can. As a recent victim of tools breaking underneath me, let's PLEASE take this seriously. It's just about the worst thing we can do to people. Look, I hate past me more than anyone, but I have to live with his idiotic, short-sighted decisions.

https://youtu.be/EjR1Ht__9KE?si=8cymBHCdN-UbPx4U - FF to 12:22

[a-mccarthy]

Hi @soltysh,

👋 from the v1.31 Communications Team! We'd love for you to opt in to write a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt in, let us know and open a Feature Blog placeholder PR against the website repository by 3rd July, 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

[soltysh]

@a-mccarthy opened https://github.com/kubernetes/website/pull/46868

[soltysh]

@liggitt @thockin thanks for your valuable input, I think it's important that we start having those conversations. I'll probably either open this topic again with sig-cli or even with sig-arch, so that we can discuss the potential path forward. Like I said when talking with Jordan on slack, nothing set in stone, but at the same time we shouldn't be stuck in a place that we seem to all agree is not the best one.

[codablock]

Has it been considered to implement a compatibility mode after kubectl moves from the deprecation to the removal state? Such a compatibility mode could shell out to the kustomize binary and emulate what the kubectl native kustomize integration was doing. This compatibility/emulation mode could be in deprecation phase from day one while printing HUGE warning messages about what is happening.

This way, kubectl could remove its compile-time dependency into kustomize and leave the compatibility mode for a much longer time. Of course, users would be required to install kustomize along kubectl, but that might be an acceptable tradeoff compared to all scripts breaking immediately.

[koba1t]

Hi! I'm currently subproject lead for kustomize.

I have some comments on the Motivation section of this proposal, and I am writing here because that proposal was merged before I checked. Could you consider updating it?

The current kubernetes release cycle doesn't match that of kustomize, oftentimes risking users of kubectl to work with outdated version of kustomize.

I feel we are able to make a kustomize release cycle to match what Kubernetes is using. In my memory, the reason the current release cycle is not regular is because we did not discuss whether it is necessary or not. I think we need to add more details about any technical or non-technical problems you feel and why you set a non-goal to change the release cycle.

"some of the kustomize dependencies has already been problematic to the core kubernetes project" needs more details ... I wasn't aware of ongoing problematic dependencies here

I agree with @liggitt's opinion. In my memory, I didn't notice any related issues, and I tried to clean up dependencies at any time. Did you want to say about kustomize's dependencies in unwanted-dependencies file in k/k?

current kubectl maintainers feel that promoting one tool over the other should not be the role of the project.

I completely agree with your opinion. Currently, we can find many manifest management tools like helm, kpt, cdk8s, cue, jsonnet, and more hundred tools.

So, I'm able to agree with your idea to remove kustomize from kubectl to improve the maintainability of both projects. But, I thought it would be better if you updated it and clearly explained why you wrote this proposal before writing the blog post.

[Princesso]

Hey @Princesso we'll probably want to put together a blog post around 1.31 release, to more advertise the fact of this deprecation along with the future plan for removal. So that more users are aware of this fact. I'll followup with appropriate PRs.

Hi @soltysh, by this comment, I am assuming that this enhancement does not need any updates to the Docs. Please correct me if I am wrong.

If it does indeed need documentation updates, please follow the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT. Also, take a look at Documenting for a release to get yourself familiarised with the docs requirement for the release. Thank you!

NB: Doc updates are different from blog posts.

[soltysh]

Hi @soltysh, by this comment, I am assuming that this enhancement does not need any updates to the Docs. Please correct me if I am wrong.

That is correct.