EKS Windows Nodes

[ofiliz]

EKS Windows worker nodes to run Windows containers.

Update – 10/8/2019 Amazon EKS now fully supports Windows containers and Windows worker nodes. https://github.com/aws/containers-roadmap/issues/69#issuecomment-539641916

Get started by looking at the EKS documentation: https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html

[emohammad]

How can i get istio installed? I am getting failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address

[mikestef9]

GA availability for EKS Windows will be released before 1.11 is no longer supported. This issue will be updated as soon as we make the GA announcement.

[bcmedeiros]

@mikestef9 thanks for that! Where can we find the most up-to-date windows node AMIs? should we still use the 1.11 ones?

[mikestef9]

Amazon EKS now fully supports adding Windows nodes as worker nodes and scheduling Windows containers.

Windows workloads are supported with Amazon EKS clusters running Kubernetes version 1.14 or later.

Please note that as of today, at least one Linux worker node is required in the cluster to support Windows node and container networking. We recommend two for high availability.

Learn how to configure a cluster for Windows support and add Windows workers nodes in the EKS documentation.

[mikestef9]

Interested to hear any feedback so far on EKS Windows support. Feel free to leave comments on this issue. Thanks!

[bcmedeiros]

@mikestef9 my path trying to use Windows worker nodes is being pretty rocky so far...

Firstly I bumped into a issue where I could not have IPs assigned to my pods: https://github.com/weaveworks/eksctl/issues/1512

After manually creating another routing table and finally overcoming this first issue, I kinda had Windows nodes working, but then I started having Linux pods being scheduled to windows nodes (yeah, this is documented, but it's very hard to set node selectors to all my deployments, including "internal" ones such as ingress and other things), so I decided to put a taint and bumped into another (it seems) eksctl bug: https://github.com/weaveworks/eksctl/issues/1590

Manually tainting the node (and also tolerate the taints in windows pods) seems to be working, but I cannot roll out this to production until everything is fully automated, so I guess I have to wait a little bit until someone at eksctl look into my reported issues.

[rparsonsbb]

@mikestef9 we're using EKS mixed OS clusters in production right now.

Aside from some hiccups with the vpc-resource-controller evicting itself after writing 500GB of logs (might not be windows related but I am working on pulling logs/opening aws support case) my BIGGEST gripe hands down is that none of the tooling that AWS provides surrounding EKS supports mixed OS clusters right now.

New node termination handler: https://github.com/aws/aws-node-termination-handler/issues/8

Container Insights: https://github.com/aws/containers-roadmap/issues/503

This is certainly more widespread than just AWS and I understand that some of this can be provided b the OSS community but I'd really like to see more support surrounding mixed OS EKS, not just spitting out AMIs that interact with the EKS masters.

[mikestef9]

@brunojcm I escalated the Windows node group taints to eksctl team and they have created a Pull Request that will land in the next eksctl release.

@rparsonsbb thanks for opening those issues. We will continue to work on improving tooling in the Kubernetes Windows ecosystem.

[bcmedeiros]

@mikestef9 thanks for that! I got the notification there, I was wondering if you had helped it to happen anyhow :)