Kustomize's embedded openapi data should cover a wide range of Kubernetes releases

[KnVerey]

During the release preparation process today, we considered bumping the openapi version bundled with Kustomize, and decided against it even though we're still bundling 1.21 and Kustomize will soon be embedded in kubectl 1.27.

The reason is that a lot of longstanding beta versions of highly used APIs were removed in v1.22-1.26. Knowing from community surveys that many organizations are running older versions of k8s and may still be using those groups, we felt ceasing to support those versions would be unnecessarily disruptive to our users. It would cost us nothing to use a an openapi list that contained the superset of the resources in the older and current releases, and our conclusion is that we should move to do this instead.

In practice this probably means enhancing the scripts we use to compile the embedded document (and modifying the output of kustomize openapi version).

/area openapi /triage accepted /kind feature

[natasha41575]

For some additional context, kustomize does have the functionality in-place to be able to support building in multiple versions of the openapi data, allowing the user to choose between them using the openapi.version field, and at one point we did do this, but the feedback we received when we went to integrate with kubectl was that they did not want to vendor so many large documents with duplicate definitions.

[natasha41575]

It has occurred to me that this issue as it is might be difficult to address since we now fetch the schema in protobuffer format.I don't know how easy it is to write a script to compile multiple protobuffer documents into one. Maybe there are tools to do it, but I don't personally have much experience with proto.

But we can also consider switching back to JSON if we bump kube-openapi, as the performance issues with the JSON parser appears to have been addressed: https://github.com/kubernetes/kube-openapi/issues/315. Switching back to JSON would also be helpful for https://github.com/kubernetes-sigs/kustomize/issues/5123. If we switch back to JSON, then we should be able to write a script that pulls multiple kubernetes versions of openapi data and merges them together.

[natasha41575]

I think for this issue, we need to do the following.

1. Investigate if we can switch back to JSON. This would involve reverting https://github.com/kubernetes-sigs/kustomize/pull/4568, bumping the kube-openapi library to pick up the fix for https://github.com/kubernetes/kube-openapi/issues/315, and verifying with benchmark testing that doing so will not result in a performance regression.

2. If the steps in (1) are ok, i.e. there won't be any performance regression, we need need to update the openapi pulling scripts to pull a wider range of Kubernetes releases and merge the openapi JSON files from different versions together.

Alternatively, we can consider trying to support several different unmerged openapi data files (then we can keep the format in proto). But, we will need buy-in from kubectl and kubernetes/kubernetes to do that, as in the past we have received pushback as they did not want to vendor in multiple large openapi data files into the upstream repository.

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted