After EKS Pod is bound to EIP, it cannot use EIP to connect to the Internet.

[weiwuduzui]

After the EKS Pod is bound to the EIP, it cannot use the EIP as the egress IP. It still uses the NAT gateway IP. What is going on?

[turmind]

The probable reason is that the NodeGroup is in a private subnet, causing this issue. The NodeGroup needs to be in public subnets, meaning the Route Table associated with the subnet where the NodeGroup resides needs to have a 0.0.0.0/0 route entry pointing to the IGW.

[jlehtimaki]

I'm having the same sort of issue. The egress IP is the nodes public IP and not the EIP that the pod is given.

 kubectl get pods <retracted> -o=custom-columns=NAME:.metadata.name,STATUS:.status.phase,PODIP:.status.podIP,EIP:.metadata.labels.aws-pod-eip-controller-public-ip \

NAME                          STATUS    PODIP             EIP
<retracted>   Running   10.0.101.76   xxx.xxx.xxx.142

root@<retracted>:~# curl icanhazip.com
xxx.xxx.xxx.219

╰─  k get nodes -o wide |grep 219
ip-10-0-101-8.eu-west-1.compute.internal     Ready    <none>   4d1h    v1.29.3-eks-ae9a62a   10.0.101.8     xxx.xxx.xxx.219   Amazon Linux 2   5.10.218-208.862.amzn2.x86_64   containerd://1.7.11

[turmind]

By default, this is indeed the case, traffic to a destination outside of the VPC has the source Pod IP SNAT'ed to the instance ENI's primary IP address. You can try:

kubectl set env daemonset -n kube-system aws-node AWS_VPC_K8S_CNI_EXTERNALSNAT=true 

To allow Pods to access the outside directly through the attached EIP. Reference link: https://docs.aws.amazon.com/eks/latest/userguide/external-snat.html

However, it should be noted that this may affect the PODs deployed in the public subnet without attached EIP being unable to access the network externally. If the EIP-Controller is deployed in the public subnet, EC2 and STS need to be added to the VPC's private-link endpoint.

[jlehtimaki]

By default, this is indeed the case, traffic to a destination outside of the VPC has the source Pod IP SNAT'ed to the instance ENI's primary IP address. You can try:

kubectl set env daemonset -n kube-system aws-node AWS_VPC_K8S_CNI_EXTERNALSNAT=true 

To allow Pods to access the outside directly through the attached EIP. Reference link: https://docs.aws.amazon.com/eks/latest/userguide/external-snat.html

However, it should be noted that this may affect the PODs deployed in the public subnet without attached EIP being unable to access the network externally. If the EIP-Controller is deployed in the public subnet, EC2 and STS need to be added to the VPC's private-link endpoint.

Yes that did the trick, thank you!

[afreyermuth98]

This is working for me also but I'm also using Cilium in my cluster and for some reasons it breaks some of my applications. Would it be possible to specify on the pod level to use the EIP to get out and node on the node / addon level ? 🤔