search

aws / secrets-store-csi-driver-provider-aws

The AWS provider for the Secrets Store CSI Driver allows you to fetch secrets from AWS Secrets Manager and AWS Systems Manager Parameter Store, and mount them into Kubernetes pods.
Apache License 2.0
476 stars 134 forks source link

CSI secret store driver fails to create secret #294

Closed ghost closed 1 year ago

ghost commented 1 year ago

Describe the bug I have deployed csi secret store driver in my cluster and it running as deamon set. I have below servieaccount,clusterrole and clusterrolebinding setup

kubectl describe ds secrets-store-csi-driver Name: secrets-store-csi-driver Selector: app=secrets-store-csi-driver Node-Selector: kubernetes.io/os=linux Labels: app=secrets-store-csi-driver app.kubernetes.io/instance=secrets-store-csi-driver app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=secrets-store-csi-driver app.kubernetes.io/version=1.3.4 helm.sh/chart=secrets-store-csi-driver-1.3.4 helm.toolkit.fluxcd.io/name=secrets-store-csi-driver helm.toolkit.fluxcd.io/namespace=kube-system Annotations: deprecated.daemonset.template.generation: 1 meta.helm.sh/release-name: secrets-store-csi-driver meta.helm.sh/release-namespace: kube-system Desired Number of Nodes Scheduled: 8 Current Number of Nodes Scheduled: 8 Number of Nodes Scheduled with Up-to-date Pods: 8 Number of Nodes Scheduled with Available Pods: 8 Number of Nodes Misscheduled: 0 Pods Status: 8 Running / 0 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels: app=secrets-store-csi-driver app.kubernetes.io/instance=secrets-store-csi-driver app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=secrets-store-csi-driver app.kubernetes.io/version=1.3.4 helm.sh/chart=secrets-store-csi-driver-1.3.4 Annotations: kubectl.kubernetes.io/default-container: secrets-store Service Account: secrets-store-csi-driver Containers: node-driver-registrar: Image: registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.8.0 Port: Host Port: Args: --v=5 --csi-address=/csi/csi.sock --kubelet-registration-path=/var/lib/kubelet/plugins/csi-secrets-store/csi.sock Limits: cpu: 100m memory: 100Mi Requests: cpu: 10m memory: 20Mi Liveness: exec [/csi-node-driver-registrar --kubelet-registration-path=/var/lib/kubelet/plugins/csi-secrets-store/csi.sock --mode=kubelet-registration-probe] delay=30s timeout=15s period=10s #success=1 #failure=3 Environment: Mounts: /csi from plugin-dir (rw) /registration from registration-dir (rw) secrets-store: Image: registry.k8s.io/csi-secrets-store/driver:v1.3.4 Ports: 9808/TCP, 8095/TCP Host Ports: 0/TCP, 0/TCP Args: --endpoint=$(CSI_ENDPOINT) --nodeid=$(KUBE_NODE_NAME) --provider-volume=/var/run/secrets-store-csi-providers --additional-provider-volume-paths=/etc/kubernetes/secrets-store-csi-providers --metrics-addr=:8095 --provider-health-check-interval=2m --max-call-recv-msg-size=4194304 Limits: cpu: 200m memory: 200Mi Requests: cpu: 50m memory: 100Mi Liveness: http-get http://:healthz/healthz delay=30s timeout=10s period=15s #success=1 #failure=5 Environment: CSI_ENDPOINT: unix:///csi/csi.sock KUBE_NODE_NAME: (v1:spec.nodeName) Mounts: /csi from plugin-dir (rw) /etc/kubernetes/secrets-store-csi-providers from providers-dir-0 (rw) /var/lib/kubelet/pods from mountpoint-dir (rw) /var/run/secrets-store-csi-providers from providers-dir (rw) liveness-probe: Image: registry.k8s.io/sig-storage/livenessprobe:v2.10.0 Port: Host Port: Args: --csi-address=/csi/csi.sock --probe-timeout=3s --http-endpoint=0.0.0.0:9808 -v=2 Limits: cpu: 100m memory: 100Mi Requests: cpu: 10m memory: 20Mi Environment: Mounts: /csi from plugin-dir (rw) Volumes: mountpoint-dir: Type: HostPath (bare host directory volume) Path: /var/lib/kubelet/pods HostPathType: DirectoryOrCreate registration-dir: Type: HostPath (bare host directory volume) Path: /var/lib/kubelet/plugins_registry/ HostPathType: Directory plugin-dir: Type: HostPath (bare host directory volume) Path: /var/lib/kubelet/plugins/csi-secrets-store/ HostPathType: DirectoryOrCreate providers-dir: Type: HostPath (bare host directory volume) Path: /var/run/secrets-store-csi-providers HostPathType: DirectoryOrCreate providers-dir-0: Type: HostPath (bare host directory volume) Path: /etc/kubernetes/secrets-store-csi-providers HostPathType: DirectoryOrCreate Events:

@@@@@@@@@@@@@@@@@@@@

kubectl describe serviceaccount secrets-store-csi-driver Name: secrets-store-csi-driver Namespace: kube-system Labels: app=secrets-store-csi-driver app.kubernetes.io/instance=secrets-store-csi-driver app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=secrets-store-csi-driver app.kubernetes.io/version=1.3.4 helm.sh/chart=secrets-store-csi-driver-1.3.4 helm.toolkit.fluxcd.io/name=secrets-store-csi-driver helm.toolkit.fluxcd.io/namespace=kube-system Annotations: meta.helm.sh/release-name: secrets-store-csi-driver meta.helm.sh/release-namespace: kube-system Image pull secrets: Mountable secrets: Tokens: Events:

@@@@@@@@@@@@@@@@@@@@

kubectl describe serviceaccount secrets-store-csi-driver Name: secrets-store-csi-driver Namespace: kube-system Labels: app=secrets-store-csi-driver app.kubernetes.io/instance=secrets-store-csi-driver app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=secrets-store-csi-driver app.kubernetes.io/version=1.3.4 helm.sh/chart=secrets-store-csi-driver-1.3.4 helm.toolkit.fluxcd.io/name=secrets-store-csi-driver helm.toolkit.fluxcd.io/namespace=kube-system Annotations: meta.helm.sh/release-name: secrets-store-csi-driver meta.helm.sh/release-namespace: kube-system Image pull secrets: Mountable secrets: Tokens: Events:

@@@@@@@@@@@@@@@@@@@@

kubectl describe clusterrole csi-secrets-store-provider-aws-cluster-role Name: csi-secrets-store-provider-aws-cluster-role Labels: app=secrets-store-csi-driver-provider-aws app.kubernetes.io/instance=secrets-provider-aws app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=secrets-store-csi-driver-provider-aws helm.sh/chart=secrets-store-csi-driver-provider-aws-0.3.4 helm.toolkit.fluxcd.io/name=secrets-provider-aws helm.toolkit.fluxcd.io/namespace=kube-system Annotations: meta.helm.sh/release-name: secrets-provider-aws meta.helm.sh/release-namespace: kube-system PolicyRule: Resources Non-Resource URLs Resource Names Verbs

serviceaccounts/token [] [] [create] nodes [] [] [get] pods [] [] [get] serviceaccounts [] [] [get] secrets [] [] [list,get,watch]

@@@@@@@@@@@@@@@@@@@@

kubectl describe clusterrolebinding csi-secrets-store-provider-aws-cluster-role Name: csi-secrets-store-provider-aws-cluster-role Labels: app=secrets-store-csi-driver-provider-aws app.kubernetes.io/instance=secrets-provider-aws app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=secrets-store-csi-driver-provider-aws helm.sh/chart=secrets-store-csi-driver-provider-aws-0.3.4 helm.toolkit.fluxcd.io/name=secrets-provider-aws helm.toolkit.fluxcd.io/namespace=kube-system Annotations: meta.helm.sh/release-name: secrets-provider-aws meta.helm.sh/release-namespace: kube-system Role: Kind: ClusterRole Name: csi-secrets-store-provider-aws-cluster-role Subjects: Kind Name Namespace

ServiceAccount secrets-store-csi-driver kube-system

I added list secret permission to the cluster role but deamonset logs is still showing an error

To Reproduce

Steps to reproduce the behavior:

Do you also notice this bug when using a different secrets store provider (Vault/Azure/GCP...)? Yes/No

If yes, the issue is likely with the k8s Secrets Store CSI driver, not the AWS provider. Open an issue in that repo.

Expected behavior

Environment: OS, Go version, etc.

Additional context Add any other context about the problem here.