search

kubernetes-sigs / aws-efs-csi-driver

CSI Driver for Amazon EFS https://aws.amazon.com/efs/
Apache License 2.0
710 stars 543 forks source link

Unable to deploy efs-csi-controller to Fargate to support Karpenter-provisioned EKS cluster #1100

Open Nuru opened 1 year ago

Nuru commented 1 year ago

/kind bug

What happened?

What you expected to happen?

The controller pods would be deployed to Fargate and and work without the Node component, and the Add-On would report status as "Active". As EC2 Nodes were provisioned, controller Pods would work from Fargate while Node Pods worked properly on EC2 Nodes.

How to reproduce it (as minimally and precisely as possible)?

See "What happened" above.

Anything else we need to know?:

The failure that is reported to Kubernetes comes from the efs-plugin container exiting with an error. IMHO it should not try to run on Fargate, and probably should not be deployed as part of the controller for this reason.

Environment

Please also attach debug logs to help us better diagnose

Log excerpts (each one just keeps repeating the quoted excerpt):

efs-csi-controller csi-provisioner

W0816 04:26:59.779601       1 connection.go:183] Still connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock

efs-csi-controller liveness-probe

W0816 04:27:00.989300       1 connection.go:173] Still connecting to unix:///csi/csi.sock

efs-csi-controller efs-plugin

I0816 05:54:46.413768       1 config_dir.go:63] Mounted directories do not exist, creating directory at '/etc/amazon/efs'
I0816 05:54:46.418766       1 metadata.go:63] getting MetadataService...
I0816 05:54:52.757469       1 metadata.go:71] retrieving metadata from Kubernetes API
F0816 05:54:52.773395       1 driver.go:56] could not get metadata: did not find aws instance ID in node providerID string
apenney commented 11 months ago

I also have the same issue. I would like to run the controllers on fargate, and have them attach EFS volumes to actual nodes that are then provisoned by karpenter.

z0rc commented 6 months ago

1195 isn't sufficient for Fargate support. Latest eks addon v1.7.6-eksbuild.1 sets securityContext.privileged: true for controller pods. This isn't supported by fargate nodes.

Please reopen.

z0rc commented 6 months ago

/reopen

k8s-ci-robot commented 6 months ago

@z0rc: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes-sigs/aws-efs-csi-driver/issues/1100#issuecomment-1964088423): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
z0rc commented 6 months ago

@Nuru could you reopen the ticket please?

Nuru commented 6 months ago

/reopen

It looks like the changes in #1195 were necessary, but not sufficient.

k8s-ci-robot commented 6 months ago

@Nuru: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/aws-efs-csi-driver/issues/1100#issuecomment-1967336392): >/reopen > >It looks like the changes in #1195 were necessary, but not sufficient. Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
sogos commented 4 months ago

Just fall in the same situation, can't deploy the add-on because kube-system is a fargate namespace. Same context = Karpenter + FargateCluster Will switch on the manual installation mode, but that seem a waste of time. Allow controllers to run on fargate would be great, thanks

skraga commented 4 months ago

We're facing the same issue as previous commenter

mskanth972 commented 4 months ago

Apologies for the delay in getting back. Our team is currently addressing this issue and will provide a solution soon. Thank you for your patience.

skraga commented 4 months ago

@mskanth972 is there any ETA when it will be available? I see a new 2.0.2 addon released but no option to set privileged to false for the controller.

mskanth972 commented 4 months ago

@skraga I have the PR ready, We will merge this and release in the upcoming version with Addons also. ECD will be by END of this Month.

skraga commented 3 months ago

@mskanth972 Thanks for your reply. Moreover, when we were considering the EKS addon for our use case we found out that it was not possible to set resource requests and limits.

z0rc commented 3 months ago

@mskanth972

I have the https://github.com/kubernetes-sigs/aws-efs-csi-driver/pull/1348 ready, We will merge this

PR is closed without merge and explanation.

ECD will be by END of this Month.

End of May passed, no updates on the issue. Please share what's the current state of this issue and plans to address it.

k8s-triage-robot commented 1 week ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

z0rc commented 1 week ago

/remove-lifecycle stale