search

aws / karpenter-provider-aws

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
https://karpenter.sh
Apache License 2.0
6.75k stars 951 forks source link

Karpenter does not work with Fargate for Karpenter/core modules #7187

Open billnbell3 opened 2 weeks ago

billnbell3 commented 2 weeks ago

Description

1.0.6 version

There does not appear to be a walkthrough around setting up Karpenter pods in Fargate. Managed nodes seems to be the only easy way to get this to work. I can use taints on the MNG to make sure only Karpenter runs on these nodes, but it would be cleaner to run Karpenter itself on Fargate, and then have Karpenter manage the nodes it creates.

I get DNS issues when trying to get Fargate to work. Fargate takes 5 minutes to come up and has timeouts as well.

Documentation on a working walkthrough would be awesome.

We would love to run the actual Karpenter on Fargate (2 nodes).

Also if anyone was able to run all cluster_addons in Fargate that also would be helpful!

billnbell3 commented 1 week ago

Anything?

engedaam commented 1 week ago

We have a toggle to setup karpenter with Fargate https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/#3-create-a-cluster

billnbell3 commented 5 days ago

Well we are using terraform and it does not seem at all simple to do.

engedaam commented 4 days ago

Have you looked at the notes for installing Karpenter? https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/#4-install-karpenter

billnbell3 commented 3 days ago

Of course. I have looked into it for long hours and tried to do it several times. I cannot get Fargate to work on all the other core pods. Karpenter will run and manage normally using node groups. I am trying to script this with terraform, and it is not easy to do, because the documentation for the setup will need to be reverse engineered. What is it doing when it runs in fargate mode?

  1. What is needed to run core pods in Fargate and the rest in Karpenter?
  2. It appears a service account is needed. What perms does this account need, and why do I need one?
  3. Has anyone set this up and actually had it work in 1.0.6 using Fargate from scratch ?
  4. DId you have to revert to eksctl ?