"MinIO Tenant with TLS via customer provided certificates" fails to complete tenant creation with "readObjectStart: expect { or n, but found [, error found in #10 byte of"

[celalsahin]

Expected Behavior

Creating a minio tenant following descriptions at https://github.com/minio/operator/blob/v3.0.29/docs/examples.md#minio-tenant-with-tls-via-customer-provided-certificates should result in a successfully created tenant, with customer provided certificates.

Current Behavior

Creating a minio tenant following descriptions at https://github.com/minio/operator/blob/v3.0.29/docs/examples.md#minio-tenant-with-tls-via-customer-provided-certificates results in error:

[csahin@qp08c minio]$ kubectl minio tenant info --name minios3 --namespace minios3
Error: v1.Tenant.Spec: v1.TenantSpec.ExternalCertSecret: readObjectStart: expect { or n, but found [, error found in #10 byte of ...|tSecret":[{"name":"m|..., bigger context ...|me":"minios3-creds-secret"},"externalCertSecret":[{"name":"minio-tls-cert","type":"kubernetes.io/tls|...
[csahin@qp08c minio]$

Also even using the provided example file at https://github.com/minio/operator/blob/v3.0.29/examples/tenant-with-custom-cert-encryption-disabled.yaml results in a very similar error (same error basically with the variables from example.yaml)

[csahin@qp08c minio]$ kubectl minio tenant info --name minios3 --namespace minios3
Error: v1.Tenant.Spec: v1.TenantSpec.ExternalCertSecret: readObjectStart: expect { or n, but found [, error found in #10 byte of ...|tSecret":[{"name":"m|..., bigger context ...|me":"minios3-creds-secret"},"externalCertSecret":[{"name":"minio-custom-cert-no-encryption-cluster-c|...
[csahin@qp08c minio]$

Possible Solution

Steps to Reproduce (for bugs)

1. Create new namespace & switch to it:

   oc new-project minios3
   oc project minios3

2. Create self signed certs & add them as secrets: (per description at https://github.com/minio/operator/blob/v3.0.29/docs/examples.md#minio-tenant-with-tls-via-customer-provided-certificates)

   
   openssl req -new -newkey rsa:2048 -sha256 -days 3650 -nodes -x509 -subj "/C=DE/ST=Bayern/L=Muenchen/O=Siemens AG/OU=SI/CN=minios3-console.minios3.svc.cluster.local/emailAddress=celal.sahin@siemens.com" -keyout minios3-console.minios3.svc.cluster.local.key -out minios3-console.minios3.svc.cluster.local.crt

openssl req -new -newkey rsa:2048 -sha256 -days 3650 -nodes -x509 -subj "/C=DE/ST=Bayern/L=Muenchen/O=Siemens AG/OU=SI/CN=minio.minios3.svc.cluster.local/emailAddress=celal.sahin@siemens.com" -keyout minio.minios3.svc.cluster.local.key -out minio.minios3.svc.cluster.local.crt

openssl req -new -newkey rsa:2048 -sha256 -days 3650 -nodes -x509 -subj "/C=DE/ST=Bayern/L=Muenchen/O=Siemens AG/OU=SI/CN=*.minios3.minios3.svc.cluster.local/emailAddress=celal.sahin@siemens.com" -keyout _wildcard.minios3.minios3.svc.cluster.local.key -out _wildcard.minios3.minios3.svc.cluster.local.crt

openssl req -new -newkey rsa:2048 -sha256 -days 3650 -nodes -x509 -subj "/C=DE/ST=Bayern/L=Muenchen/O=Siemens AG/OU=SI/CN=*.minios3-hl.minios3.svc.cluster.local/emailAddress=celal.sahin@siemens.com" -keyout _wildcard.minios3-hl.minios3.svc.cluster.local.key -out _wildcard.minios3-hl.minios3.svc.cluster.local.crt

oc create secret tls console-tls-cert --key="minios3-console.minios3.svc.cluster.local.key" --cert="minios3-console.minios3.svc.cluster.local.crt" oc create secret tls minio-buckets-cert --key="_wildcard.minios3.minios3.svc.cluster.local.key" --cert="_wildcard.minios3.minios3.svc.cluster.local.crt" oc create secret tls minio-hl-cert --key="_wildcard.minios3-hl.minios3.svc.cluster.local.key" --cert="_wildcard.minios3-hl.minios3.svc.cluster.local.crt" oc create secret tls minio-tls-cert --key="minio.minios3.svc.cluster.local.key" --cert="minio.minios3.svc.cluster.local.crt"

2. Per https://github.com/minio/operator/tree/v3.0.29#tenant-setup create the tenant.yaml file via:
kubectl minio tenant create --name minios3 --servers 4 --volumes 16 --capacity 160Gi --namespace minios3 --storage-class local-storage -o > tenant.yaml

3. Edit the tenant.yaml so it includes self-signed certs as described at https://github.com/minio/operator/blob/v3.0.29/docs/examples.md#minio-tenant-with-tls-via-customer-provided-certificates - you can find attached resulting tenant.yaml (renamed to log as it github wont attach yaml)
[tenant.log](https://github.com/minio/operator/files/5952604/tenant.log)

4. Run the create command & observe that it fails with above mentioned error message

[csahin@qp08c minio]$ oc create -f tenant.yaml tenant.minio.min.io/minios3 created secret/minios3-creds-secret created secret/minios3-console-secret created [csahin@qp08c minio]$ kubectl minio tenant info --name minios3 --namespace minios3 Error: v1.Tenant.Spec: v1.TenantSpec.ExternalCertSecret: readObjectStart: expect { or n, but found [, error found in #10 byte of ...|tSecret":[{"name":"m|..., bigger context ...|me":"minios3-creds-secret"},"externalCertSecret":[{"name":"minio-tls-cert","type":"kubernetes.io/tls|... [csahin@qp08c minio]$

## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
I would like to setup with custom certificates that I provide

## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include minio-operator version caused the regression, if you have these details. -->

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio-operator`): VERSION: v3.0.29
* Environment name and version (e.g. kubernetes v1.17.2): OCP Cluster version is 4.6.13
* Server type and version: 
* Operating System and version (`uname -a`):
* Link to your deployment file: 
[tenant.log](https://github.com/minio/operator/files/5952631/tenant.log)

[Alevsk]

@celalsahin Hi, this seems to be an issue with the current CRD you are using, can you please share the output of kubectl get crd tenants.minio.min.io -o yaml command? It seems that the version of Operator you are currently using has a CRD that expects an object to be passed in the v1.TenantSpec.ExternalCertSecret field but you are passing an array

[celalsahin]

Hi,

Just that you know how I got to this point/minio operator version - I just installed it via kubectl krew & I have 3.0.29, at least according to krew

[csahin@qp08c ~]$ kubectl krew info minio
NAME: minio
INDEX: default
URI: https://github.com/minio/operator/releases/download/v3.0.29/kubectl-minio_linux_amd64.zip
SHA256: efa9d6d91f2f746758ca700c070bf0e6ae390337a843a1e39c578ff2ead51653
VERSION: v3.0.29
HOMEPAGE: https://github.com/minio/operator/tree/master/kubectl-minio
DESCRIPTION:
The kubectl-minio plugin wraps the MinIO Operator and provides a simplified
interface to create and manage MinIO tenant clusters.

CAVEATS:
\
 | * For resources that are not in default namespace, currently you must
 |   specify -n/--namespace explicitly (the current namespace setting is not
 |   yet used).
/
[csahin@qp08c ~]$

Here is the output for your question attached crd.log , as it is quite long.

Regards Celal

[nitisht]

Please use latest minio operator release 4.0.6 and let us know how it goes