search

redhat-cop / cert-utils-operator

Set of functionalities around certificates packaged in a Kubernetes operator
Apache License 2.0
95 stars 35 forks source link

Duplicated CA certs cause "ExtendedValidationFailed" #89

Closed traceygithub closed 3 years ago

traceygithub commented 3 years ago

Venafi issues a certificate which includes the CA cert in spec.tls.certificate. After enabling cert-utils-operator.redhat-cop.io/certs-from-secret on a route, it causes a validation error (ExtendedValidationFailed) since there are now two CA certs in the chain.

As a work round, we would have to manually disable cert-utils-operator.redhat-cop.io/certs-from-secret and remove spec.tls.caCertificate, then the route works fine.

cert-utils operator should either a) remove spec.tls.caCertificate if it exists or b) move the CA cert from spec.tls.certificate to spec.tls.caCertificate.

raffaelespazzoli commented 3 years ago

@traceygithub thanks for this issue. let me see if I understand correctly. Your venafi puts the root ca cert both in the tls.crt (here together with the entire validation chain and the top level certificate) and in the ca.crt. so now the root level certificate is present two times in cert material that the route publishes to a client. Then when you connect with a client your client fails with extended validation failed. Is that what is happening?

traceygithub commented 3 years ago

@raffaelespazzoli that is right: the Venafi puts the CA cert in both tls.crt and ca.crt. The route gets rejected from Openshift (not from a client) due to "ExtendedValidationFailed" with two CA certs found in the route TLS. The case is similar to the one reported in tnozicka/openshift-acme#68 for kubernetes.io/tls-acme.

$ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD parksmap ExtendedValidationFailed parksmap https edge None

raffaelespazzoli commented 3 years ago

@mathianasj can you take a look at this? I know we have several Venafi users, why are they not all seeing the same problem? Could it be a Venafi side configuration?

daniyalj commented 3 years ago

Im running into the same problem while testing Venafi (SaaS) with OpenShift.

The route is getting rejected. And I see the value caCertificate at the end of certificate again

daniyalj commented 3 years ago

@traceygithub did you find the solution to the problem? I do see spec.tls.caCertificate separately as well as part of spec.tls.certificate

EDIT 1 : Removing tls.spec.caCertificate solves the "rejected" error

EDIT 2: Not sure if its a Venafi issue. Using Ingress objects with cert-manager ignores the caCertificate on the same secret

traceygithub commented 3 years ago

@daniyalj From the Venafi side, there is an option to set "IncludeChain" false. If the cert-manager can choose this option when requesting a certificate, I think the solution would be easy.

raffaelespazzoli commented 3 years ago

@traceygithub is it possible to set that option by default in venafi for the certificates requested by cert-manager?

daniyalj commented 3 years ago

thanks @traceygithub are you using Venafi TPP or Venafi Cloud?

traceygithub commented 3 years ago

@daniyalj we are using Venafi TPP.

traceygithub commented 3 years ago

@raffaelespazzoli here are some thoughts: 1) on the Venafi side, the "IncludeChain" option is available for all CSRs, not just for the cert-manager. 2) changes on the cert-manager targeting for Kubernetes would impact wider range of audiences. 3) cert-utils operator is mainly used for Openshift route object, would it be an easy solution to have additional annotation/option which does not publish tls.ca to the route if users require, like for this case?

daniyalj commented 3 years ago

@raffaelespazzoli any update on this? thanks!

raffaelespazzoli commented 3 years ago

@daniyalj here you can find a fix: https://github.com/raffaelespazzoli/cert-utils-operator/tree/fix%2389 would you be able to test it.

daniyalj commented 3 years ago

@raffaelespazzoli just tested it with Venafi Cloud and it is working fine. I will do a test with our TPP instance as well and report back. This is the route created:

apiVersion: v1
items:
- apiVersion: route.openshift.io/v1
  kind: Route
  metadata:
    annotations:
      cert-utils-operator.redhat-cop.io/certs-from-secret: weather-tls
      cert-utils-operator.redhat-cop.io/inject-CA: false
    labels:
      app: weather-app
      app.kubernetes.io/component: weather-app
      app.kubernetes.io/instance: weather-app
    name: weather
    namespace: venafi-test
  spec:
    host: weather.apps.cluster-3725.***.com
    port:
      targetPort: 8080-tcp
    tls:
      certificate: |
        -----BEGIN CERTIFICATE-----
        *****
        -----END CERTIFICATE-----

        -----BEGIN CERTIFICATE-----
        ****
        -----END CERTIFICATE-----

        -----BEGIN CERTIFICATE-----
        ******
        -----END CERTIFICATE-----
      insecureEdgeTerminationPolicy: None
      key: |
        -----BEGIN RSA PRIVATE KEY-----
        ******
        -----END RSA PRIVATE KEY-----
      termination: edge
    to:
      kind: Service
      name: weather-app
      weight: 100
    wildcardPolicy: None
      host: weather.apps.cluster-3725.*****.com
      routerCanonicalHostname: apps.cluster-3725.****.com
      routerName: default
      wildcardPolicy: None
faisalrazzak commented 3 years ago

Adding information for future reference: