search

aws / amazon-vpc-cni-k8s

Networking plugin repository for pod networking in Kubernetes using Elastic Network Interfaces on AWS
Apache License 2.0
2.28k stars 743 forks source link

eks with secondary network for pods #3072

Open davirezendegb opened 1 month ago

davirezendegb commented 1 month ago

Following the documentation, we end up in the following scenario. https://aws.github.io/aws-eks-best-practices/networking/custom-networking/

The pod receives the IPs from the secondary subnet and to try to access resources outside the VPC, the main IP of the node in the main network is used, but for resources in the same VPC it tries to use the secondary IP that the pod receives. Is there a way to mask the pod IP to the main IP for internal resources in the same VPC?

pod ip: 100.127.35.58 node ip: 192.168.1.44

when trying to access an RDS resource in the same vpc the pod uses the ip 100.127.35.58

to access a resource outside the vpc a masking is done so that the pod uses the node ip 192.168.1.44

We would like this masking to also be done for internal resources in the same VPC

orsenthil commented 1 month ago

Is there a way to mask the pod IP to the main IP for internal resources in the same VPC?

No, there is not a way to mask the pod ip to the main ip (or node ip) of internal resources in the same VPC.

If you use the hostnetworking pod, then the ip of the pod is the same as the node's primary ip.

davirezendegb commented 1 month ago

do we not have the possibility to create a function for this? "AWS_VPC_K8S_CNI_INTERNALSNAT=true"

orsenthil commented 1 month ago

Do you mean, create a new flag to have an Internal SNAT? I am not sure how that will work with the property of hostnetwork pods.

davirezendegb commented 1 month ago

the intention is to make the pods receive the ips of the secondary subnet as expected, but to communicate internally use the ip of the main subnet that is also tied to the node Captura de Tela_Área de Seleção_20241015174143 for example, the pods receive the range 100.127.xxx.xxx and intermanete they use this ip 100.127.xxx.xxx to communicate with an RDS.

the idea is to use the primary ip of the node: 10.72.xxx.xxx (this occurs for when the resource is outside the vpc)

davirezendegb commented 1 month ago

in GCP and azure there is this way of performing this configuration

https://cloud.google.com/kubernetes-engine/docs/how-to/ip-masquerade-agent

https://github.com/Azure/ip-masq-agent-v2

orsenthil commented 2 weeks ago

@davirezendegb - can you raise this feature request containers-roadmap. It needs to be evaluated and prioritized as a product requirement for CNI.