SNAT IP configurable by IP Pool instead of being set globally

[clementchambault]

Explanation: We are working in an environment with the host having multiple IPs on the same interface. Issue 2222 https://github.com/projectcalico/calico/issues/2222 change the possibility to switch from MASQUERADE to SNAT by setting the variable FELIX_NATOUTGOINGADDRESS. We would like to have the possibility to set a different source IP for the SNAT depending on the Pods from where the traffic is coming from.

Expected Behavior

When creating an IP Pool instead of just setting the natoutgoing to true, add optionnaly a source IP to be used

Current Behavior

By default MASQUERADE is used and you can change it to a SNAT source IP address used by traffics coming from all pods:

Possible Solution

Add an optional field when defining a pool of IP adress: apiVersion: projectcalico.org/v3 kind: IPPool metadata: name: my.ippool-1 spec: cidr: 10.1.0.0/16 ipipMode: CrossSubnet natOutgoing: true SNATIP: "x.x.x.x"

Steps to Reproduce (for bugs)

1. 2. 3. 4.

Context

For the embedded world for hosting applications and work with a different node IP

Your Environment

Calico: 3.22.1 K3S: 1.20.6+k3s1

[caseydavenport]

Could you share a bit more information on your use case? Is this to integrate with external firewalls or something like that?

Normally that would require full-fledged proxies, since SNAT will only work for addresses that are actually present on the node. Meaning, depending on where the pod lands you'll end up with a different SNAT IP, no matter the pool.

[clementchambault]

Yes it is to integrate with external firewalls (and simulate as if there are multiple different system whereas in reality there is only one) It is an embedded platform providing services to hosted applications (running on same cluster inside different namespace). We would like to have a specific IP for the pods providing services (IP of the node) and one or many IP addresses for hosted applications (IPs added to the same interface).

Currently our use case is to have one node so no issue with that. In the future our uses case is to have a static node allocation of the pods against the IPs added to the node for the hosted applications. (mechanism to reallocate the IP if a node fails).

Thanks

[caseydavenport]

Currently our use case is to have one node so no issue with that.

I don't think we can design a feature that requires a single node cluster (or even static scheduling to a specific node) in order to function, so we would need to do something a bit more complex than this.

You can use IP pools to allocate pod addresses to specific deployments in a controlled way - that might get you what you need? Then you could use the real pod IPs and have your firewalls match on CIDR?

[caseydavenport]

For a single node, you can also assign a real IP address to a pod using an annotation and ensure that the pod always has that address. Assuming your network can route the pod IPs, that might get you what you need.

Otherwise, I'm not sure how we can do this in a generic enough way to support multi-node clusters.

[clementchambault]

Ok assigning a pod real IP is not really what what we want, we would like to let the current automatic IP allocation for the pods, but just manage the source IP for external traffic. Furthermore we would like to have multiple Pods having same source IP address (same as multiple services running on the same equipment).

One node: IP addresses are automatically assigned to the host interface prior starting kubernetes Multi node: IP addresses are managed by metalLB in layer 2 mode that takes care of assigning the IP addresses to the correct nodes. (PODs are assigned to correct associated node using node affinity)

My current working solution (but require manual modification of iptables rules, not really good to do with calico at the same time): -Disable MASQUERADE and SNAT for PODS -Use Calico in append mode for the rules -Use multiple pool IP -manually add snat iptables rules based on source IP pool subnet