search

aws / secrets-store-csi-driver-provider-aws

The AWS provider for the Secrets Store CSI Driver allows you to fetch secrets from AWS Secrets Manager and AWS Systems Manager Parameter Store, and mount them into Kubernetes pods.
Apache License 2.0
454 stars 127 forks source link

GRPC error: failed to mount secrets store object #29

Closed Gearheads closed 3 years ago

Gearheads commented 3 years ago

Hello:

We are currently experiencing an issue with the Secrets Store CSI Driver v0.0.23, where it is not able to mount the secret store object. We are seeing the Pods are stuck in a "ContainerCreating" state:

pod/robtest-aws-secret-manager-deploy-77747544cf-826n5   0/1     ContainerCreating   0          74m
pod/robtest-aws-secret-manager-deploy-77747544cf-8m9cd   0/1     ContainerCreating   0          74m

Here is the SecretProviderClass YAML:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: robtest-aws-secret-provider-class
spec:
  provider: aws
  parameters:
    region: us-east-2
    objects: |
        - objectName: "arn:aws:secretsmanager:us-east-2:<AWS-account-#>:secret:MySecret-3jq4EL"
          objectType: "secretsmanager"

Any ideas as to why we are experiencing this issue below?

I0616 18:17:11.646404       1 nodeserver.go:300] "Using grpc client" provider="aws" 
  pod="robtest-aws-secret-manager-deploy-77747544cf-6pfvr"

I0616 18:19:11.643100       1 nodeserver.go:73] "unmounting target path as node publish volume failed" 
  targetPath="/var/lib/kubelet/pods/945bf1d1-a91b-4018-accd-1d79f5a1dbf9/volumes/kubernetes.io~csi/secrets-store-inline/mount"
  pod="cluster-addons/robtest-aws-secret-manager-deploy-77747544cf-6pfvr"

E0616 18:19:11.664341       1 utils.go:79] GRPC error: failed to mount secrets store objects for
  pod cluster-addons/robtest-aws-secret-manager-deploy-77747544cf-6pfvr,
  err: rpc error: code = Canceled desc = context canceled
Gearheads commented 3 years ago

Turns out the issue was missing proxy values within the csi-secrets-store-provider-aws daemonset:

spec:
  serviceAccountName: csi-secrets-store-provider-aws
  containers:
    - name: provider-aws-installer
       imagePullPolicy: Always
       env:
       - name: HTTP_PROXY
         value: http://<proxy-url>:<proxy-port>
       - name: HTTPS_PROXY
         value: http://<proxy-url>:<proxy-port>
       - name: NO_PROXY
         value: <no-proxy-list>