aws / containers-roadmap

This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
https://aws.amazon.com/about-aws/whats-new/containers/
Other
5.21k stars 317 forks source link

[EKS] [Feature]: Enable DNS64 in CoreDNS on IPv6 clusters #2352

Closed johngmyers closed 3 months ago

johngmyers commented 4 months ago

Community Note

Tell us about your request

Allow the CoreDNS EKS addon to be easily configured to enable the dns64 plugin on IPv6 clusters.

Even better would be if the dns64 plugin was enabled by default in IPv6 clusters.

The coredns ConfigMap should look something like:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    eks.amazonaws.com/component: coredns
    k8s-app: kube-dns
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
          }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        dns64
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }

Which service(s) is this request for? CoreDNS EKS addon

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

We run IPv6 EKS clusters with dual-stack nodes. Our container registry is IPv4-only and reachable from our nodes' private IP space.

We want our nodes to always pull images over IPv4, avoiding NAT Gateway transit fees for image pulls. If DNS64 is enabled for a node's subnet, the node could get a NAT64 address for the container registry and pull an image through the NAT gateway, incurring unnecessary cost.

We want to enable the dns64 plugin in CoreDNS so that pods (at least those with dnsPolicy: ClusterFirst) get DNS64 behavior, but disable DNS64 for the subnet so the node itself does not get DNS64 behavior.

Are you currently working around this issue?

We are currently enabling DNS64 in the node subnets and paying any resulting unnecessary costs.

Additional context

For what it's worth, kOps enables this for all IPv6 clusters.

Attachments

johngmyers commented 3 months ago

I had not been aware of the node-level 169.254.172.0/22 NATing done by Amazon VPC CNI, which significantly decreases the need to do NAT64/DNS64 in IPv6 clusters with dual-stack nodes. I'm going to close this as having limited utility at the current time.