How to use a wildcard DNS and certs?

[rberrelleza]

Is there a way to configure kubeprod to use a pre-created wildcard DNS entry and cert for all the services deployed?

Since all the services deployed will have the same subdomain (e.g. services.example.com), having a single DNS entry for .services.example.com and a single cert for .services.example.com would speed up service provisioning a lot, instead of the 1-3 minutes it takes today.

[falfaro]

That's a very good question.

At the moment, they way BKPR enables ingress support for Grafana, Kibana and Prometheus is by requesting one X.509 certificate from Let's Encrypt for each of them (3 certificates in total). This is handled by the cert-manager pod. At the same time, the external-dns pod will configure three A and TXT records for these ingress resources.

If I understand you correctly, you would like BKPR to behave different than this: you already have a wildcard DNS entry and X.509 certificate for your entire domain. Please correct me if I understood it wrong.

[rberrelleza]

I have two different ideas in mind:

• The first one is what you mention. I already have a certificate and a DNS entry, and I want all the services on my cluster that use the nginx-ingress and external DNS to just use it

• I don't have a cert or a DNS entry, and I want BKPR to generate it for me (via letsencrypt) and renew it. Same for the DNS entry.

We currently do this manually ( we have a script to copy the certificate between namespaces and all that) but it would be great to have it taken care of.

The cert-manager and nginx-ingress teams discussed this and decided not to implement it, but this might be a problem that makes more sense to fix at BKPR's level.

[anguslees]

All of this is possible, with appropriate overrides to the manifests :)

For the first case (you already have DNS and certs), you can just create the DNS entry and k8s Secret containing the TLS secret, and external-dns and cert-manager should see they're already there and not override them (I haven't tested this).

If you want to provide these values through the override mechanism instead of managing them through a different tool, here's how to do that through BKPR/jsonnet:

// This is kubeprod-manifest.jsonnet 
local kube = import "kube.libsonnet";
(import "https://releases.kubeprod.io/files/v1.1.0/manifests/platforms/gke.jsonnet") {
   config:: import "kubeprod-autogen.json",

   // Place your overrides here
   grafana+: {
      local grafana = self,

      ingress+: {
         metadata+: {
            annotations+: {
               "kubernetes.io/tls-acme": "false",  // turn off letsencrypt for this Ingress
               "external-dns.alpha.kubernetes.io/controller": "noexist", // turn off external-dns for this Ingress
            },
         },
         host: "grafana.example.com",  // or whatever hostname.  Assumed to already exist.
      },

      // Create a new Secret containing a TLS key from local crt/key files
      mycert: kube.Secret(grafana.ingress.spec.tls[0].secretName) {
         type: "kubernetes.io/tls",
         data_: {
           "tls.crt": importstr "mycert.crt",  // PEM file, read during jsonnet expansion
           "tls.key": importstr "mycert.key",  // (ditto)
         },
      },
   },
}

I think your second case is what BKPR does out of the box (manage DNS and cert for you). If I missed something, and it isn't obvious how to do that from looking at the above example, please follow up.

(I would be overjoyed if someone could verify the above actually works (heh!), and send a PR that adds the above question/answer to a doc or faq.)

[arapulido]

@rberrelleza were you able to test the suggestion above?

[rberrelleza]

Not yet @arapulido . I'm configuring a cluster this week, I'll try the suggestion.

[arapulido]

@rberrelleza any updates?

[arapulido]

@rberrelleza Hello! did you finally test this?

[ghost]

I did, It didn't work. I think using grafana in the example is confusing. this is better to demonstrate with a new resource.