poseidon / typhoon

Minimal and free Kubernetes distribution with Terraform
https://typhoon.psdn.io/
MIT License
1.94k stars 322 forks source link

CoreOS: validatingWebhookConfiguration: failed to load system roots and no roots provided; open /etc/ssl/certs/ca-certificates.crt: permission denied #1518

Open JordanP opened 2 weeks ago

JordanP commented 2 weeks ago

Description

I try to create a CustomResource, for which there's a validating_webhook_configuration. When I kubectl apply I get

Internal error occurred: failed calling webhook "vpooler.cnpg.io": failed to call webhook: 
Post "https://cnpg-webhook-service.cnpg-system.svc:443/validate-postgresql-cnpg-io-v1-pooler?timeout=10s": 
tls: failed to verify certificate: x509: failed to load system roots and no roots provided; open 
/etc/ssl/certs/ca-certificates.crt: permission denied

Steps to Reproduce

CoreOS, Typhoon 1.31. I am trying to deploy a CNPG cluster. The installation of the CRDs and Operator works fine, but when I try to create the following resource:

resource "kubernetes_manifest" "cluster" {
  depends_on = [module.yavin]
  manifest = {
    apiVersion = "postgresql.cnpg.io/v1"
    kind       = "Cluster"
    metadata = {
      name      = "cluster-example"
      namespace = "default"
    }
    spec = {
      instances = 3
      storage = {
        size = "1Gi"
      }
    }
  }
}

I get this image

Expected behavior

I should be able to create that resource.

Environment

Possible Solution It seems the cert volumes (/etc/ssl/certs seems to be a symlink to /etc/pki) are mounted in the kube-api-server:

    Mounts:
      /etc/kubernetes/pki from secrets (ro)
      /etc/pki from etc-pki (ro)
      /etc/ssl/certs from etc-ssl (ro)

so it should work, but I am not sure it actually works.

JordanP commented 1 week ago

I think the key part to reproduce this is caBundle of webhook.clientConfig.service needs to be an empty string.

(the CNPG operator has some logic to fill in that caBundle if it detects it's empty and will generate a self-signed certificate)