search

knative / serving

Kubernetes-based, scale-to-zero, request-driven compute
https://knative.dev/docs/serving/
Apache License 2.0
5.57k stars 1.16k forks source link

Unreachable validating webhook when using external/virtual control plane #15376

Closed MathiasPius closed 4 months ago

MathiasPius commented 4 months ago

Ask your question here:

Is it possible to configure the validating webhook to run as an externally reachable NodePort service instead of an internal one?

Workarounds

I'm aware I can of course just set up a separate NodePort service pointing at the webhook, but the validation.webhook.serving.knative.dev will still specify the internal service as the target, which means quite a bit of hacking to get it working, and upgrades would likely break everything all over again.

I see there is limited ability to override webhook container configuration in the KnativeServing CRD, but I can't find anything to let me modify the service and/or ValidatingWebhookConfiguration.

Reasoning

I'm installing knative on a Kubernetes 1.29.1 cluster which is using an external/virtual/managed control plane (like EKS, GKE, etc.) where the kube-apiserver is running on a server that is not part of the cluster. As a result, the api server is not technically part of the CNI networking overlay and is therefore unable to reach internal pod and service IPs.

There are similar issues documented for cert-manager here: https://cert-manager.io/docs/concepts/webhook/#known-problems-and-solutions

MathiasPius commented 4 months ago

Looks like I can work around this by setting up Konnectivity.