emissary-ingress / emissary

open source Kubernetes-native API gateway for microservices built on the Envoy Proxy
https://www.getambassador.io
Apache License 2.0
4.34k stars 680 forks source link

TLSContext auto-reloading of certs? #4076

Open OmegaVVeapon opened 2 years ago

OmegaVVeapon commented 2 years ago

Describe the bug Using Ambassador 1.13.10, new certificates are not automatically reloaded.

To Reproduce Steps to reproduce the behaviour:

  1. Create a TLSContext that load certificates via files like so:
      # https://www.getambassador.io/reference/core/tls/
      apiVersion: getambassador.io/v2
      kind:  TLSContext
      name:  tls-private
      cert_chain_file: "/mnt/secrets-store/my-cert"
      private_key_file: "/mnt/secrets-store/my-cert"
      hosts: ["*.foo.bar.com", "*.foo.bar.com:443"]
      alpn_protocols: h2, http/1.1
      min_tls_version: v1.2
      cipher_suites:
        - "[ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384]"
        - "[ECDHE-RSA-AES128-GCM-SHA256|ECDHE-RSA-CHACHA20-POLY1305|ECDHE-RSA-AES256-GCM-SHA384]"
      ecdh_curves:
        - X25519
        - P-256
      redirect_cleartext_from: 8080
  2. Verify that you're able to resolve the hosts when hitting Ambassador via HTTPS with the provided certificate.
  3. Delete/replace the certificate file in the container

Expected behavior Ambassador should detect a change was done to the paths defined in cert_chain_file and/or private_key_file. If the file was changed, the TLSContext should negotiate the TLS handshake with the new cert. If it was removed, new calls should start failing. Instead, nothing changes.

Versions (please complete the following information):

Additional context Having Ambassador reload the TLSContext certs is critical to us in order to avoid SSL certificate expiration errors without having to perform manual rolling-restarts for our various Ambassador deployments. The closest thing I found to this issue was https://github.com/emissary-ingress/emissary/issues/2936 but that seems mostly related to Istio so opened this one instead.

OmegaVVeapon commented 2 years ago

FWIW, we ended up going back to using secret since the cert_chain_file/private_key_file method never worked for us and that made Ambassador start reloading the certs in Envoy when they were updated again.

This difference in reloading mechanisms between secret-based vs file-based certs is incredibly significant and should probably be documented in the TLSContext section... shouldn't it?

aman955 commented 2 years ago

Hi @OmegaVVeapon Can you please share the details of the ambassador configuration which you were using that made the it work for you. For me replacing the cert-chain with secrets is causing the mTLS to fail.