EBS CSI Driver - deployed with incorrect trust relationship

[daveschmidt86]

Describe the bug

Update (6/21//2022):

In further troubleshooting, it was discovered that the Addon is deploying an incorrect trust relationship with the OIDC sub as:

system:serviceaccount:kube-system:aws-ebs-csi-driver

When I manually update this to match the ebs-csi documentation (https://docs.aws.amazon.com/eks/latest/userguide/csi-iam-role.html) it works, here is the correct sub:

system:serviceaccount:kube-system:ebs-csi-controller-sa

EBS CSI Addon Version: 1.6.2 K8 Cluster version: 1.21

When attempting to create a new volume, PVC is reporting the following auth error:

"failed to provision volume with StorageClass "ebs-sc": rpc error: code = Internal desc = Could not create volume "pvc-93e1b143-5f66-4ddd-85fb-6bd293e4d866": could not create volume in EC2: WebIdentityErr: failed to retrieve credentials caused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity status code: 403, request id:"

using example Dynamic Provisioning manifests from: https://docs.aws.amazon.com/eks/latest/userguide/ebs-sample-app.html

Expected Behavior

PVC and volume should have created

Current Behavior

Auth Error assuming Web Identity

Reproduction Steps

Stood up new cluster, specified k8 version 1.21, ebs csi addon version 1.6.2.

Cluster and default gp2 strorage class verified and created by default.

Deploy example Dynamic Provisioning manifests from this doc: https://docs.aws.amazon.com/eks/latest/userguide/ebs-sample-app.html

Possible Solution

Modify the sub in the OIDC trust relationship for the ebs-csi service account from:

system:serviceaccount:kube-system:aws-ebs-csi-driver

to this:

system:serviceaccount:kube-system:ebs-csi-controller-sa

Additional Information/Context

No response

CDK CLI Version

2.20

EKS Blueprints Version

1.0.1

Node.js Version

16

Environment details (OS name and version, etc.)

MACOS

Other information

No response

[elamaran11]

This issue is fixed now with this merge, Please validate and close the issue.

[rstrazza]

When will 1.0.5 be released? Assuming the fix would be part of it. I'm experiencing the same error.

Btw, thanks for the workaround, daveschmidt86!

[shapirov103]

We are finalizing the quality assurance for the next batch of changes (1.1.0 release) targeting early next week.

[shapirov103]

@daveschmidt86 this is fixed in 1.1.0, if you get a chance, please verify.

[shapirov103]

@daveschmidt86 closing the issue, we can reopen if you discover any defective behavior.