With IPIP enabled, traffic between Calico hosts and pods should be allowed without nat-outgoing

[tmjd]

See projectcalico/calicoctl#1296 for history of this request.

As @matthewdupre said a fix would be to:

Adjust the routing with --ipip, using multiple routing tables and ip rules to do source routing (essentially); using tunl0 for all traffic where the source or destination is a Calico pod.

And a use case that @matthewdupre reported:

Here's a use-case from fxposter on Slack: running Kubernetes and host endpoints in multi-AZ AWS.

• The pods need to be tunnelled, but it's undesirable to use nat-outgoing to the host endpoints, because then they can't properly enforce policy.

[tmjd]

This should not be isolated to Host Endpoints but any host running calico/node and part of the Calico cluster.

[caseydavenport]

Just did some playing around with this on GCE. My setup was something like this:

Host A - IP 10.128.0.53 Host B - IP 10.128.0.17

Host A has a single pod running on it, with IP 192.168.227.129 Host B has a host endpoint + some network policy on it which selects the aforementioned pod.

With some config on the pod's' host (Host A) I was able to get traffic from pod->hostB to take the ipip tunnel.

Config on Host A

$ ip rule show
0:      from all lookup local 
32765:  from 192.168.227.129 to 10.128.0.17 lookup 5 
32766:  from all lookup main 
32767:  from all lookup default 

$ ip route show table 5
10.128.0.17 dev tunl0  scope link 

I then just needed to disable outgoing NAT.

I suspect we could make this more generic with something like this:

ip rule add from <node's /26> to <node subnet> table 5

ip route add <node subnet> dev tunl0

The last bit would be how to make this play nice with nat-outgoing. We'd probably need to make it aware that it shouldn't NAT traffic from a pod->node. Otherwise the traffic will take the IPIP tunnel, but will sill end up being NAT'd anyway.