kubectl-minio plugin "Error: Unauthorized" with KUBECONFIG env variable set

ol88 commented 3 years ago

I have two Kubernetes cluster with 2 separate kubeconfig files, my KUBECONFIG environment variable is set and contains the path to both kubeconfig files:

$ echo $KUBECONFIG
/home/user/.kube/config-kubernetes-dev:/home/user/.kube/config-kubernetes:

The current-context is set to the correct cluster where minio is successfully deployed via kubectl minio init

$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://kubernetes.example.com:6443
  name: kubernetes
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://kubernetes.example.com:6443
  name: kubernetes-dev
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
- context:
    cluster: kubernetes-dev
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes-dev
current-context: kubernetes-admin@kubernetes-dev
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

Note: I never have both clusters running at the same time which is why they share the same url in the config. Both are test clusters running on the same hardware.

All kubectl generic commands including minio plugin commands kubectl minio init and kubectl minio delete work perfectly. However, none of kubectl minio tenant or kubectl minio proxy commands work, they all result in:

$ kubectl minio tenant list
Error: Unauthorized

using the --kubeconfig flag also does not work:

$ kubectl minio tenant list --kubeconfig /home/user/.kube/config-kubernetes-dev
Error: Unauthorized

The only way to make it work is to change the KUBECONFIG environment variable to a single kubeconfig file path.

$ export KUBECONFIG=/home/user/.kube/config-kubernetes-dev:
$ kubectl minio tenant list

Tenant 'tenant', Namespace 'minio-tenant', Total capacity 40 MiB

  Current status: Provisioning Console Deployment
  MinIO version: minio/minio:RELEASE.2021-06-17T00-10-46Z
  Console version: minio/console:v0.7.5

Expected Behavior

kubectl minio tenant and kubectl proxy should succeed when multiple config path are specified in KUBECONFIG environment variable as long as the kubectl current-context is correctly set.

Current Behavior

kubectl minio tenant and kubectl minio proxy both fail with error Error: Unauthorized when multiple config path are specified in KUBECONFIG environment variable and the kubectl current-context is correctly set.

Possible Solution

Not familiar enough with the code sorry.

Steps to Reproduce (for bugs)

Create a single kubernetes cluster
Duplicate the cluster config file to make a secondary kubeconfig file for a different fictional cluster as per above.
Set KUBECONFIG environment variable to point to both kubeconfig file paths.
Make sure the "current-context" of kubectl config is properly set to the live cluster context created in 1.
Initialize minio operator on live cluster with the following command: kubectl minio init
Try to open the proxy kubectl minio proxy -n minio-operator
It should fail with Error: Unauthorized...

Context

See above

Regression

Unknown

Your Environment

Version used (minio-operator): v4.1.3
Environment name and version (e.g. kubernetes v1.17.2): kubernetes v1.21.3
Server type and version:
Operating System and version (uname -a): Ubuntu 18.04.4 LTS
Link to your deployment file: kubectl-minio plugin installed using krew

ol88 commented 3 years ago

After posting this issue it just started working again with both path set in the env variable. Yet I was successfully able to reproduce this bug multiple times before.

I suggest someone tries to reproduce, if it cannot be reproduced in another environment this could be closed.

ol88 commented 3 years ago

Nope it's back to not working.

harshavardhana commented 3 years ago

Version used (minio-operator): v4.1.3

Please upgrade to kubectl-minio v4.2.8 and let us know if it works.

ol88 commented 3 years ago

Sorry for the late reply, just got around to try with the most recent version.

I updated from 4.1.3 to 4.2.14 (latest version) using krew. Update was successful but the behaviour describe above is unchanged. Still getting Error: Unauthorized for either kubectl minio tenant or kubectl minio proxy.
I then tried fully deleting minio from the cluster and re-initializing a new operator (v4.2.14). Same result.

So unfortunately, recent updates since 4.1.3 have not addressed this problem.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

minio / operator