digitalocean / Kubernetes-Starter-Kit-Developers

Hands-on tutorial and Automation stack for an operations-ready DigitalOcean Kubernetes (DOKS) cluster.
761 stars 263 forks source link

Certificates not renewed automatically #201

Closed jmd9019 closed 1 year ago

jmd9019 commented 2 years ago

Hi, I am using nginx ingress and have deployed cert manager and ingress following this tutorial https://www.youtube.com/watch?v=MpovOI5eK58&list=RDCMUCaPX53JLxxSbwZz_Ra_cL0g&index=2

Its been almost 5-6 months since deployed I was expecting as per tutorial my certificates to be renewed automatically which didn't happen and my staging environment certificates are expired and need help with the renewal also where did I go wrong? Also I need to fix this because I have deployed in same way to my production clusters also, so need help ASAP

v-ctiutiu commented 2 years ago

Hi @jmd9019

Sorry to hear that. As far as I know cert-manager should renew certificates automatically, unless there's an issue somewhere down the chain.

Is it possible to send more information about your current setup for staging environment such as:

  1. Kubernetes version.
  2. Cert-manager version.
  3. Nginx ingress version.

Next, is it possible to send relevant configuration such as:

  1. Current Ingress resource configuration.
  2. Current configuration for cert-manager's Issuer resource.

Also, it would be helpful to see event information emitted by cert-manager's Issuer resource. I'm interested if any relevant messages are sent by the issuer. On top of that, logs emitted by cert-manager controller Pods is a plus as well.

Note: Make sure to strip sensitive information from manifests and logs, if any.

Thanks.

jmd9019 commented 2 years ago

Hi @v-ctiutiu, It was needed urgently so as other comments in StackOverflow suggested I changed the challenge from http01 to dns01 which renewed my certificates as of now, but don't know if I might again face same issue with dns01 challenge also One thing I remember is that for http01 it was creating HTTP acme endpoint which I think DO loadbalancer was redirecting to https which was giving 404 error

v-ctiutiu commented 1 year ago

@jmd9019

Interesting. Another thing to remember is the rate limit that Let's Encrypt is imposing. But, shouldn't be the case because you're renewing certificates, hence doesn't apply (as stated in their rate limit documentation page).

Another thing to check and keep in mind is the duplicate certificates limit. In case of environments that deal a lot with frequent deployments (such as development envs), or where you perform a lot of testing, I think it's best to use the Let's Encrypt staging environment.

I think most people use wildcard certificates in the end, hence the dns-01 challenge option. This way, you avoid requesting too many certificates at once for multiple exposed services, and maybe avoid tricky situations such as rate limits or whatever Let's Encrypt is imposing. Another thing to take into consideration is how to perform this process gradually and stay on the safe side so to speak.

In the end, it's advised to find the root cause so that you are not hit by this issue again. If you have logging or the observability stack enabled (and I assume you do), check if you are affected by the above mentioned possible cause(s).

Hope it helps.

jmd9019 commented 1 year ago

Hi @v-ctiutiu,
Thanks for the update I suspect that we had this issue 'Another thing to check and keep in mind is the duplicate certificates limit. In case of environments that deal a lot with frequent deployments (such as development envs), or where you perform a lot of testing, I think it's best to use the Let's Encrypt staging environment.' which you mentioned as we are continuously deploying to production new changes also I was using Let's encrypt production environment

jmd9019 commented 1 year ago

Closing ticket as of now will reopen again if this occurs