Multi TLS support - losesing own internal certificate

[sergey-kudriavtsev]

When using an external certificate, the tenant loses its own internal certificate (making it unusable)

You also have to use MINIO_SERVER_URL, although the documentation states that it is no longer required env minio

Expected Behavior

X509v3 Subject Alternative Name: 
                DNS:main-tenant-pool-0-{0...3}.main-tenant-hl.default.svc.cluster.local, DNS:minio.default.svc.cluster.local, DNS:minio.default, DNS:minio.default.svc, DNS:*.main-tenant-hl.default.svc.cluster.local, DNS:*.default.svc.cluster.local

Current Behavior

The domain test.example.con is given as an example

X509v3 Subject Alternative Name: 
                DNS:*test.example.con, DNS:*.s3.example.con

Possible Solution

1. Add a field to the tenant operator for manual set path
2. Automatically read the common domain name from certificates and form a mountPath from it

Manual Fix Tempsolution

1. Don`t use additionalVolumes

2. Use mount cert like this

   ...
   additionalVolumes:
   - name: s3-example-com-tls
     secret:
       secretName: s3.example.com-tls
       items:
         - key: tls.crt
           path: public.crt
         - key: tls.key
           path: private.key
   
   additionalVolumeMounts:
   - name:  s3-example-com-tls
     mountPath: /tmp/certs/s3.example.com
   ...

Steps to Reproduce (for bugs)

1. Create cert secret (or request by cert-manager) s3.example.com-tls
2. Add in tenant external cert

   apiVersion: minio.min.io/v2
   kind: Tenant
   metadata:
     name: main-tenant
     namespace: default
   spec:
        externalCertSecret:
           name: s3.example.comt-tls

3. create tenant
4. list dir in pod

   ls -ls /tmp/certs/

5. check tls

   kubectl run tets-ssl --image=alpine/openssl --rm -ti -n default --command -- openssl s_client -showcerts -connect minio.s3-mainnet.svc:443 </dev/null 2>/dev/null | openssl x509 -inform pem -text | grep -A1 "Subject Alternative Name"
   

   Context

   How it works is described here Minio Description TLS But the code generates a path without taking the domain into account Code link This makes it unacceptable to use embedded certificates with external or multi-external ones.

Regression

Yes

Your Environment

• Version used (minio-operator): 6.0.3
• Environment name and version (e.g. kubernetes v1.17.2): kubernetes v1.30.2

[cesnietor]

we'll try to reproduce and share our findings.

[allanrogerr]

I was not able to reproduce with the following steps. Please retest using v6.0.4

Apply operator

kubectl kustomize github.com/minio/operator\?ref=v6.0.4 | kubectl apply -f -
kubectl apply -k github.com/minio/operator/examples/kustomization/base

Create secrets

kubectl -n minio-tenant create secret tls minio-tenant-external-tls-ui --cert=miniohq-allan-3-public-ui.crt --key=miniohq-allan-3-private-ui.key  
kubectl -n minio-tenant create secret tls minio-tenant-external-tls-api --cert=miniohq-allan-3-public-api.crt --key=miniohq-allan-3-private-api.key 
kubectl patch tenant -n minio-tenant myminio --type='merge' -p '{"spec":{"externalCertSecret": [{"name": "minio-tenant-external-tls-ui","type": "kubernetes.io/tls"}, {"name": "minio-tenant-external-tls-api", "type": "kubernetes.io/tls"}] }}'

Validate tenant spec reflects changes

Validate minio pods contained all expected certs

kubectl --namespace minio-tenant exec -it myminio-pool-0-0 -- /bin/sh
sh-5.1$ ls -la /tmp/certs 
total 4
drwxrwsrwt 3 root 1000  180 Oct 29 10:33 .
drwxrwxrwt 1 root root 4096 Oct 29 10:33 ..
drwxr-sr-x 5 root 1000  140 Oct 29 10:33 ..2024_10_29_10_33_28.2552440600
lrwxrwxrwx 1 root 1000   32 Oct 29 10:33 ..data -> ..2024_10_29_10_33_28.2552440600
lrwxrwxrwx 1 root 1000   10 Oct 29 10:33 CAs -> ..data/CAs
lrwxrwxrwx 1 root 1000   17 Oct 29 10:33 hostname-1 -> ..data/hostname-1
lrwxrwxrwx 1 root 1000   17 Oct 29 10:33 hostname-2 -> ..data/hostname-2
lrwxrwxrwx 1 root 1000   18 Oct 29 10:33 private.key -> ..data/private.key

Create openssl pod

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: openssl
  namespace: default
  labels:
    app: openssl
spec:
  containers:
  - image: alpine/openssl
    command:
      - "sleep"
      - "604800"
    imagePullPolicy: IfNotPresent
    name: openssl
  restartPolicy: Always
EOF

kubectl --namespace default exec -it openssl -- /bin/sh

Validate server presented the required certs

/ # openssl s_client -showcerts -connect minio.minio-tenant.svc:443 </dev/null 2>/dev/null | openssl x509 -inform pem -text | grep -A1 "Subject Alternative Name"
            X509v3 Subject Alternative Name: 
                DNS:myminio-pool-0-{0...3}.myminio-hl.minio-tenant.svc.cluster.local, DNS:minio.minio-tenant.svc.cluster.local, DNS:minio.minio-tenant, DNS:minio.minio-tenant.svc, DNS:*.myminio-hl.minio-tenant.svc.cluster.local, DNS:*.minio-tenant.svc.cluster.local
/ # openssl s_client -showcerts -connect miniohq-allan-3.k1.min.dev:443 </dev/null 2>/dev/null | openssl x509 -inform pem -text | grep -A1 "Subject Alternative Name"
            X509v3 Subject Alternative Name: 
                DNS:miniohq-allan-3.k1.min.dev
/ # openssl s_client -showcerts -connect console.miniohq-allan-3.k1.min.dev:443 </dev/null 2>/dev/null | openssl x509 -inform pem -text | grep -A1 "Subject Alternative Name"
            X509v3 Subject Alternative Name: 
                DNS:console.miniohq-allan-3.k1.min.dev