Open doctorpangloss opened 2 years ago
@doctorpangloss could you share a small guide on how you managed to connect on-prem nodes to EKS?
Can you tell me about the current progress. I have same problem with you when add edge node to EKS Cluster. @doctorpangloss
@vectornguyen76 @Tonkonozhenko here you go - https://appmana.com/blog/hacking-aws-on-premises-eks-nodes
@vectornguyen76 @Tonkonozhenko here you go - https://appmana.com/blog/hacking-aws-on-premises-eks-nodes
Provisioning and retaining EC2 instances so that you can impersonate them on your external workers is a super-ugly hack, not really a solution.
Provisioning and retaining EC2 instances
On the one hand, if you are trying to use on premises nodes, I can see that it's disappointing that you'll need at least one EC2 instance anywhere.
On the other hand, if you are trying to use AWS without ever "provisioning and retaining" an EC2 instance, well, you better start believing in EC2 instances, because nearly everything in AWS is made more useful by them. You will only need one of the very cheapest possible instance to admit all your nodes, and it can be a preexisting EKS node which you will have anyway to run CoreDNS.
You will only need one of the very cheapest possible instance to admit all your nodes
Do you not need one EC2 instance per on-prem node for each kubelet to impersonate? Even if you can get away with this hack using only a single node, it's clearly a hack, and there's zero guarantee that your cluster doesn't break one day due to changes committed to the AWS provider. Not viable for production IMO.
You will only need one of the very cheapest possible instance to admit all your nodes
Do you not need one EC2 instance per on-prem node for each kublet to impersonate?
You only need at most one EC2 instance ID, that one ID will work for all your on premises nodes / it can appear multiple times on multiple workers' arguments.
Of course this is a hack, but I happily used it in production for years. It's the end user's prerogative. The limitations of how the control plane can reach your nodes is much more impactful.
Community Note
Tell us about your request
Better support on-premises nodes connecting to the AWS control plane.
Which service(s) is this request for? EKS
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? The objective is to join on-premises nodes to the EKS control plane. This makes it easier to use EKS to meet demand through scaling; and it allows sophisticated users to work around out-of-date hardware and software on AWS.
Are you currently working around this issue? I am currently using on-premises nodes with EKS with some degraded capabilities, like limitations on the control plane being able to access the on-premises nodes, and the flimsiness of pretending to be an actual AWS instance.
Additional context You already support this technologically, you just need to align your product teams with this.