TLS certificates are overwritten with CA certificates when both `secretName` and `caSecretName` are set

[Pela2silveira]

Describe the bug

using kind: RabbitmqCluster and set both spec.tls.secretName and spec.tls.caSecretName, the operator set the same value for both files, being this value the CA cert.

To Reproduce

Steps to reproduce the behavior:

1. this are the manifests I have deploy (I am using cert manager and nginx controller)

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
   annotations:
   external-dns.alpha.kubernetes.io/hostname: rabbitmq.appstest.com.ar
   cert-manager.io/cluster-issuer: letsencrypt-production
   name: rabbitmq-test
   namespace: rabbitmq-test
   spec:
   ingressClassName: nginx
   tls:
   - hosts:
       - rabbitmq.appstest.com.ar
     secretName: rabbitmq-appstest.tls
   rules:
   - host: rabbitmq.appstest.com.ar
     http:
       paths:
         - pathType: Prefix
           path: "/"
           backend:
             service:
               name: rabbitmq
               port:
                 number: 15672
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
   name: tcp-services
   namespace: ingress-nginx
   data:
   5671: "rabbitmq-test/rabbitmq:5671"
   ---
   #this is for the client cert
   apiVersion: cert-manager.io/v1
   kind: Issuer
   metadata:
   name: rabbitmq-selfsigned-issuer
   namespace: rabbitmq-test
   spec:
   selfSigned: {}
   ---
   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
   name: rabbitmq-test-ca
   namespace: rabbitmq-test
   spec:
   isCA: true
   commonName: rabbitmq.appstest.com.ar
   subject:
   organizations:
     - ACME Inc.
   organizationalUnits:
     - Widgets
   secretName: rabbitmq-test-ca-secret
   privateKey:
   algorithm: ECDSA
   size: 256
   duration: 175200h
   issuerRef:
   name: rabbitmq-selfsigned-issuer
   kind: Issuer
   group: cert-manager.io
   ---
   apiVersion: cert-manager.io/v1
   kind: Issuer
   metadata:
   name: rabbitmq-test-issuer
   namespace: rabbitmq-test
   spec:
   ca:
   secretName: rabbitmq-test-ca-secret
   ---
   apiVersion: rabbitmq.com/v1beta1
   kind: RabbitmqCluster
   metadata:
   name: rabbitmq
   namespace: rabbitmq-test
   spec:
   replicas: 1
   tls:
   secretName: rabbitmq-appstest.tls
   caSecretName: rabbitmq-test-ca-secret
   rabbitmq:
   additionalConfig: |
     ssl_options.fail_if_no_peer_cert = true

2. set this flag in nginx-ingress controller:

   args:
   - /nginx-ingress-controller
   - --tcp-services-configmap=ingress-nginx/tcp-services

   reference:

3. Exec sh in pod and check cert files in /etc/rabbitmq-tls folder
4. If in RabbitmqCluster, I set only secretName it works, but I can't use mlts:

   apiVersion: rabbitmq.com/v1beta1
   kind: RabbitmqCluster
   metadata:
   name: rabbitmq
   namespace: rabbitmq-test
   spec:
   replicas: 1
   tls:
   secretName: rabbitmq-appstest.tls
   rabbitmq:
   additionalConfig: |
     ssl_options.fail_if_no_peer_cert = true

   Expected behavior /etc/rabbitmq-tls/tls.crt and /etc/rabbitmq-tls/ca.crt should be different, the first one is the server cert, the second is the CA used to validate client certificate.

Other details

-----BEGIN CERTIFICATE-----
xxxxx
-----END CERTIFICATE-----

rabbitmq@rabbitmq2-server-0:/$ cat /etc/rabbitmq-tls/ca.crt 

-----BEGIN CERTIFICATE-----
xxxxx
-----END CERTIFICATE-----```

being xxxxx the same content...

## Version and environment information

- RabbitMQ: 3.13.1
- RabbitMQ Cluster Operator: 2.8.0
- Kubernetes: v1.27.2
- Cloud provider or hardware configuration: running in OKE

[github-actions[bot]]

This issue has been marked as stale due to 60 days of inactivity. Stale issues will be closed after a further 30 days of inactivity; please remove the stale label in order to prevent this occurring.

[Zerpet]

Removing stale label, this report is still relevant

[Zerpet]

I confirm this is a legitimate bug. Here are simpler repro manifests:

cert-secrets.yaml

```yaml --- apiVersion: v1 kind: Secret metadata: name: rabbitmq-appstest.tls type: kubernetes.io/tls stringData: ca.crt: | -----BEGIN CERTIFICATE----- foobar= -----END CERTIFICATE----- tls.crt: | -----BEGIN CERTIFICATE----- totallyfake= -----END CERTIFICATE----- tls.key: | -----BEGIN RSA PRIVATE KEY----- yep/legitkey== -----END RSA PRIVATE KEY----- --- apiVersion: v1 kind: Secret metadata: name: rabbitmq-test-ca-secret type: kubernetes.io/tls stringData: ca.crt: | -----BEGIN CERTIFICATE----- IamTheCoolCA/BOWBEFOREME11111= -----END CERTIFICATE----- tls.crt: | -----BEGIN CERTIFICATE----- ImIrrelevant= -----END CERTIFICATE----- tls.key: | -----BEGIN RSA PRIVATE KEY----- EvenMoreIrrelevant== -----END RSA PRIVATE KEY----- ```

rabbitmq.yaml

```yaml --- apiVersion: rabbitmq.com/v1beta1 kind: RabbitmqCluster metadata: name: rabbitmq spec: replicas: 1 tls: secretName: rabbitmq-appstest.tls caSecretName: rabbitmq-test-ca-secret ```

The expected certificate contents are:

Expected contents

__tls.crt__ ``` -----BEGIN CERTIFICATE----- totallyfake= -----END CERTIFICATE----- ``` __tls.key__ ``` -----BEGIN RSA PRIVATE KEY----- yep/legitkey== -----END RSA PRIVATE KEY----- ``` __ca.crt__ ``` -----BEGIN CERTIFICATE----- IamTheCoolCA/BOWBEFOREME11111= -----END CERTIFICATE----- ```

The actual outcome is that only CA secret certificates are present. We may be able to project individual keys in the mount path, but I have to catch up with volume projections in Kubernetes.