4.17.X Patched images cert expiration date only valid for 9 days

[adrianriobo]

In order to test 4.17.0-ec.1 the patched images were created to extend the cert validity for a year on 2nd of July

But when we run test with the bundle created with the patched images the cert expiration date was set to 10th of July:

E0711 09:27:25.303299   10580 memcache.go:265] couldn't get current server API group list: Get "https://api.crc.testing:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-07-11T09:27:25Z is after 2024-07-10T05:27:06Z
Unable to connect to the server: tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-07-11T09:27:25Z is after 2024-07-10T05:27:06Z

[praveenkumar]

Bundle created for 4.17.0-ec.1 along with patched image doesn't have some certs validated for an year most kubeapi one.

$ echo -e "NAMESPACE\tNAME\tEXPIRY" && oc get secrets -A -o go-template='{{range .items}}{{if eq .type "kubernetes.io/tls"}}{{.metadata.namespace}}{{" "}}{{.metadata.name}}{{" "}}{{index .data "tls.crt"}}{{"\n"}}{{end}}{{end}}' | while read namespace name cert; do echo -en "$namespace\t$name\t"; echo $cert | base64 -d | openssl x509 -noout -enddate; done | column -t | grep 2024
NAMESPACE       NAME    EXPIRY
openshift-config-managed                          kube-controller-manager-client-cert-key             notAfter=Jul  21  12:28:17  2024  GMT
openshift-config-managed                          kube-scheduler-client-cert-key                      notAfter=Jul  21  12:28:16  2024  GMT
openshift-kube-apiserver-operator                 aggregator-client-signer                            notAfter=Jul  22  12:50:52  2024  GMT
openshift-kube-apiserver                          aggregator-client                                   notAfter=Jul  22  12:50:52  2024  GMT
openshift-kube-apiserver                          check-endpoints-client-cert-key                     notAfter=Jul  21  12:27:56  2024  GMT
openshift-kube-apiserver                          control-plane-node-admin-client-cert-key            notAfter=Jul  21  12:28:16  2024  GMT
openshift-kube-apiserver                          external-loadbalancer-serving-certkey               notAfter=Jul  21  12:27:58  2024  GMT
openshift-kube-apiserver                          internal-loadbalancer-serving-certkey               notAfter=Jul  21  12:27:55  2024  GMT
openshift-kube-apiserver                          kubelet-client                                      notAfter=Jul  21  12:27:56  2024  GMT
openshift-kube-apiserver                          localhost-serving-cert-certkey                      notAfter=Jul  21  12:27:57  2024  GMT
openshift-kube-apiserver                          service-network-serving-certkey                     notAfter=Jul  21  12:27:55  2024  GMT
openshift-kube-controller-manager                 kube-controller-manager-client-cert-key             notAfter=Jul  21  12:28:17  2024  GMT
openshift-kube-scheduler                          kube-scheduler-client-cert-key                      notAfter=Jul  21  12:28:16  2024  GMT
openshift-operator-lifecycle-manager              pprof-cert                                          notAfter=Jul  17  10:55:53  2024  GMT

[praveenkumar]

I think https://github.com/openshift/cluster-kube-apiserver-operator/blob/release-4.17/pkg/operator/certrotationcontroller/certrotationcontroller.go#L120-L129 would be culprit.

[praveenkumar]

Will try to use my private repo to test changes and see if that helps.

[praveenkumar]

So I added following patch to our internal gist-git which use to build the patch images for kube-apiserver-operator

commit fb24c0467b003a361fdccbe13d5d3851b427f8ee (HEAD -> rhaos-4.17-rhel-9, origin/rhaos-4.17-rhel-9)
Author: Praveen Kumar <kumarpraveen.nitdgp@gmail.com>
Date:   Wed Jul 17 11:12:17 2024 +0530

    remove force cert rotation every couple days for development

    This is done to test cert rotation for dev builds and if we want to
    create bundles for long cert period then this part need to be removed

    - https://github.com/openshift/cluster-kube-apiserver-operator/commit/604719e4fa3bcf72c45a2ebdb39aa57f02483862

diff --git a/pkg/operator/certrotationcontroller/certrotationcontroller.go b/pkg/operator/certrotationcontroller/certrotationcontroller.go
index 3eeb9bf18..74e62da41 100644
--- a/pkg/operator/certrotationcontroller/certrotationcontroller.go
+++ b/pkg/operator/certrotationcontroller/certrotationcontroller.go
@@ -122,10 +122,6 @@ func newCertRotationController(
                rotationDay = day
                klog.Warningf("!!! UNSUPPORTED VALUE SET !!!")
                klog.Warningf("Certificate rotation base set to %q", rotationDay)
-       } else {
-               // for the development cycle, make the rotation 60 times faster (every twelve hours or so).
-               // This must be reverted before we ship
-               rotationDay = rotationDay / 60
        }

        certRotator := certrotation.NewCertRotationController(

after creating the bundle using that image I can see now certs are valid for a year instead 9 days

$ echo -e "NAMESPACE\tNAME\tEXPIRY" && oc get secrets -A -o go-template='{{range .items}}{{if eq .type "kubernetes.io/tls"}}{{.metadata.namespace}}{{" "}}{{.metadata.name}}{{" "}}{{index .data "tls.crt"}}{{"\n"}}{{end}}{{end}}' | while read namespace name cert; do echo -en "$namespace\t$name\t"; echo $cert | base64 -d | openssl x509 -noout -enddate; done | column -t | grep 2024
NAMESPACE       NAME    EXPIRY
openshift-operator-lifecycle-manager              pprof-cert                                          notAfter=Jul  18  09:20:59  2024  GMT

[praveenkumar]

This is fixed on the internal dist git PR which remove this block.