search

buildpacks-community / kpack

Kubernetes Native Container Build Service
Apache License 2.0
946 stars 162 forks source link

Error with kpack webhook when trying to install CF Korifi #1670

Open htbg opened 3 months ago

htbg commented 3 months ago

Hi, there!

I'm trying to install Cloud Foundry Korifi in a cluster I built with kubeadm, but I'm getting this error:

helm install korifi https://github.com/cloudfoundry/korifi/releases/download/v0.12.0/korifi-0.12.0.tgz \
    --namespace="$KORIFI_NAMESPACE" \
    --set=generateIngressCertificates=true \
    --set=rootNamespace="$ROOT_NAMESPACE" \
    --set=adminUserName="$ADMIN_USERNAME" \
    --set=api.apiServer.url="api.$BASE_DOMAIN" \
    --set=defaultAppDomainName="apps.$BASE_DOMAIN" \
    --set=containerRepositoryPrefix=index.docker.io/korifi/ \
    --set=kpackImageBuilder.builderRepository=index.docker.io/korifi/kpack-builder \
    --set=networking.gatewayClass=$GATEWAY_CLASS_NAME \
    --wait
Error: INSTALLATION FAILED: failed to create resource: Internal error occurred: failed calling webhook "defaults.webhook.kpack.io": failed to call webhook: Post "https://kpack-webhook.kpack.svc:443/defaults?timeout=10s": context deadline exceeded

The kpack installation did not show any errors and the Pods seem to be running fine.

Any ideas?

ollie-kane-CB commented 3 days ago

Being able to call the webhook depends on a few things. I had struggled with this as well, due to the networking considerations in my cluster.

One is to check the Namespace created by Kpack. It adds an annotation by default which configures a Network Policy that can be triggered by your Container Network Interface. Try removing that annotation to see if things work. If so, make sure to take mitigative steps as this increases the surface area.

Likewise, if you are in certain managed environments (like I was in EKS), and you are using a non-AWS CNI (i'm using Cilium) be aware that the managed control plane cannot reach data plane webhooks. you can overcome this by setting the Webhook deployment to use Host Networking. If you do so, you'll need to pick an unused port, ensure that your firewalls on the underlying LAN allow the traffic. If you go this route, you'll also need to pass in a ENV VAR to the deployment to update the port it listens on if non-default.

There's other possible issues you can run into. Keep in mind that there's currently no Helm or other package manager like system to deploy kpack, so depending on how you're installing, you may need to fork, or keep a local copy of a Kustomize or similar template system so future updates don't mangle your custom work. I'm hoping to get time in the medium future to contribute to this repository something that fills that gap.

Best of luck!

image