search

kubernetes-sigs / vsphere-csi-driver

vSphere storage Container Storage Interface (CSI) plugin
https://docs.vmware.com/en/VMware-vSphere-Container-Storage-Plug-in/index.html
Apache License 2.0
288 stars 173 forks source link

fix kubeconfig env and flag values for CSI controller and syncer #2918

Closed divyenpatel closed 3 weeks ago

divyenpatel commented 3 weeks ago

What this PR does / why we need it: This PR is required to support KUBECONFIG env variable for vSphere CSI controller and syncer pod. when KUBECONFIG env variable is supplied but kubeconfig flag is not set, current code is setting default flag value.

This value is set by controller-runtime - https://github.com/kubernetes-sigs/controller-runtime/blame/main/pkg/client/config/config.go#L40-L58

{"level":"info","time":"2024-06-11T17:42:07.435629424Z","caller":"kubernetes/kubernetes.go:77","msg":"kubecfgFlag: &{Name:kubeconfig Usage:Paths to a kubeconfig. Only required if out-of-cluster. Value: DefValue:}","TraceId":"a4361fad-495c-4333-a49a-6eeda97edb5b","TraceId":"a3f66d1a-b463-4640-9fca-6330a8a49624"}

default value set in the kubeconfig flag is over riding env variable supplied in the container. This result into env variable being ignored, and code path is creating kubernetes client using in-cluster config instead of using kubeconfig supplied in environment variable.

Testing done:

Unit test

divyenp@divyenp0LVDQ kubernetes % go test -v -run TestGetKubeConfigPath
=== RUN   TestGetKubeConfigPath
=== RUN   TestGetKubeConfigPath/EnvVarSetFlagNotSet
{"level":"info","time":"2024-06-11T11:48:25.814524-07:00","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"/path/from/env\""}
{"level":"info","time":"2024-06-11T11:48:25.815015-07:00","caller":"kubernetes/kubernetes.go:111","msg":"Kubeconfig flag is set but empty, using environment variable value"}
{"level":"info","time":"2024-06-11T11:48:25.815026-07:00","caller":"kubernetes/kubernetes.go:121","msg":"Final Kubeconfig path used: \"/path/from/env\""}
=== RUN   TestGetKubeConfigPath/EnvVarNotSetFlagSet
{"level":"info","time":"2024-06-11T11:48:25.815328-07:00","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"\""}
{"level":"info","time":"2024-06-11T11:48:25.815337-07:00","caller":"kubernetes/kubernetes.go:109","msg":"Kubeconfig path obtained from kubeconfig flag: \"/path/from/flag\""}
{"level":"info","time":"2024-06-11T11:48:25.815342-07:00","caller":"kubernetes/kubernetes.go:121","msg":"Final Kubeconfig path used: \"/path/from/flag\""}
=== RUN   TestGetKubeConfigPath/EnvVarSetFlagSet
{"level":"info","time":"2024-06-11T11:48:25.815551-07:00","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"/path/from/env\""}
{"level":"info","time":"2024-06-11T11:48:25.815559-07:00","caller":"kubernetes/kubernetes.go:109","msg":"Kubeconfig path obtained from kubeconfig flag: \"/path/from/flag\""}
{"level":"info","time":"2024-06-11T11:48:25.815564-07:00","caller":"kubernetes/kubernetes.go:121","msg":"Final Kubeconfig path used: \"/path/from/flag\""}
=== RUN   TestGetKubeConfigPath/NeitherEnvVarNorFlagSet
{"level":"info","time":"2024-06-11T11:48:25.815601-07:00","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"\""}
{"level":"info","time":"2024-06-11T11:48:25.81561-07:00","caller":"kubernetes/kubernetes.go:111","msg":"Kubeconfig flag is set but empty, using environment variable value"}
{"level":"info","time":"2024-06-11T11:48:25.815616-07:00","caller":"kubernetes/kubernetes.go:119","msg":"No Kubeconfig path found, either from environment variable or flag"}
=== RUN   TestGetKubeConfigPath/EnvVarSetFlagEmpty
{"level":"info","time":"2024-06-11T11:48:25.815648-07:00","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"/path/from/env\""}
{"level":"info","time":"2024-06-11T11:48:25.815657-07:00","caller":"kubernetes/kubernetes.go:111","msg":"Kubeconfig flag is set but empty, using environment variable value"}
{"level":"info","time":"2024-06-11T11:48:25.815662-07:00","caller":"kubernetes/kubernetes.go:121","msg":"Final Kubeconfig path used: \"/path/from/env\""}
--- PASS: TestGetKubeConfigPath (0.00s)
    --- PASS: TestGetKubeConfigPath/EnvVarSetFlagNotSet (0.00s)
    --- PASS: TestGetKubeConfigPath/EnvVarNotSetFlagSet (0.00s)
    --- PASS: TestGetKubeConfigPath/EnvVarSetFlagSet (0.00s)
    --- PASS: TestGetKubeConfigPath/NeitherEnvVarNorFlagSet (0.00s)
    --- PASS: TestGetKubeConfigPath/EnvVarSetFlagEmpty (0.00s)
PASS
ok      sigs.k8s.io/vsphere-csi-driver/v3/pkg/kubernetes        0.857s

Verified on the live setup when kubeconfig flag is supplied, below is the log

{"level":"info","time":"2024-06-11T18:59:25.54610768Z","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"\"","TraceId":"6f14886c-e45f-4b3f-b2ac-43ea43fdda44","TraceId":"3c7f6a9b-663d-4799-b576-a99a8ce1147a"}
{"level":"info","time":"2024-06-11T18:59:25.546400074Z","caller":"kubernetes/kubernetes.go:109","msg":"Kubeconfig path obtained from kubeconfig flag: \"/mnt/volume2/kubeconfig\"","TraceId":"6f14886c-e45f-4b3f-b2ac-43ea43fdda44","TraceId":"3c7f6a9b-663d-4799-b576-a99a8ce1147a"}
{"level":"info","time":"2024-06-11T18:59:25.54643341Z","caller":"kubernetes/kubernetes.go:121","msg":"Final Kubeconfig path used: \"/mnt/volume2/kubeconfig\"","TraceId":"6f14886c-e45f-4b3f-b2ac-43ea43fdda44","TraceId":"3c7f6a9b-663d-4799-b576-a99a8ce1147a"}
{"level":"info","time":"2024-06-11T18:59:25.546439746Z","caller":"kubernetes/kubernetes.go:75","msg":"k8s client using kubeconfig from /mnt/volume2/kubeconfig","TraceId":"6f14886c-e45f-4b3f-b2ac-43ea43fdda44","TraceId":"3c7f6a9b-663d-4799-b576-a99a8ce1147a"}

Verified on the live setup when kubeconfig env variable but flag is not supplied, below is the log

{"level":"info","time":"2024-06-11T19:02:08.163661995Z","caller":"kubernetes/kubernetes.go:100","msg":"Kubeconfig path obtained from environment variable \"KUBECONFIG\": \"/mnt/volume2/kubeconfig\"","TraceId":"1a419fdc-7577-47c3-b71e-459a1198704b","TraceId":"15df45c3-9766-4cb1-96b6-5e89e03aba3b"}
{"level":"info","time":"2024-06-11T19:02:08.163997406Z","caller":"kubernetes/kubernetes.go:111","msg":"Kubeconfig flag is set but empty, using environment variable value","TraceId":"1a419fdc-7577-47c3-b71e-459a1198704b","TraceId":"15df45c3-9766-4cb1-96b6-5e89e03aba3b"}
{"level":"info","time":"2024-06-11T19:02:08.164126487Z","caller":"kubernetes/kubernetes.go:121","msg":"Final Kubeconfig path used: \"/mnt/volume2/kubeconfig\"","TraceId":"1a419fdc-7577-47c3-b71e-459a1198704b","TraceId":"15df45c3-9766-4cb1-96b6-5e89e03aba3b"}
{"level":"info","time":"2024-06-11T19:02:08.164470011Z","caller":"kubernetes/kubernetes.go:75","msg":"k8s client using kubeconfig from /mnt/volume2/kubeconfig","TraceId":"1a419fdc-7577-47c3-b71e-459a1198704b","TraceId":"15df45c3-9766-4cb1-96b6-5e89e03aba3b"}

Release note:

fix kubeconfig env and flag values for CSI controller and syncer
gohilankit commented 3 weeks ago

/approve /ok-to-test

shalini-b commented 3 weeks ago

/approve

k8s-ci-robot commented 3 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: divyenpatel, gohilankit, shalini-b

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/vsphere-csi-driver/blob/master/OWNERS)~~ [divyenpatel,gohilankit,shalini-b] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
gohilankit commented 3 weeks ago

/lgtm