Allow passthrough to other iptables chains on no match.

[bcavns01]

Expected Behavior

When none of felix's rules match, there should be an option to either jump to a specified chain, or return so that subsequent chains after calico's can continue processing.

Current Behavior

When there is no match, packets are simply dropped.

Possible Solution

https://github.com/projectcalico/felix/pull/2161 introduces the option for IptablesNotPassedAction, which gets closer to what we're looking for; but instead of only DROP or REJECT, JUMP and RETURN options would be great.

Or primary use case is related to migrating to calico on hosts where we have dockerized and non-dockerized services and thus want to run felix in insert mode. We have some rules that we can't replicate in felix at the moment, and we'd like to be able to let felix handle what it can but then jump to our "normal" chains for any exotic rules we might have.

Aside from just migrating to felix, this would also provide the general flexibility needed for future-proofing full migration/adoption. If someone finds a need for some fancy iptables rules that can't be well expressed in felix yet, this offers a path to an immediate solution that still allows felix to be the first point in the processing pipeline while waiting for felix to catch-up.

[tmjd]

Could you elaborate on:

Or primary use case is related to migrating to calico on hosts where we have dockerized and non-dockerized services and thus want to run felix in insert mode.

Because from the rest of your suggested solution it sounds like you could switch to append mode and have your fancy iptables rules before the felix rules.

Also could you provide some examples of some of the exotic rules you have and their intent, that would help us understand what is needed that Felix cannot currently do.

[bcavns01]

Thank you for taking the time to respond.

For a time, we did run with append-mode. The problem arose as we shifted away from host networking and towards docker networking. For the machines where we need to lock down access to docker containers not running with host networking, we need insert-mode so that calico's rules always come before any docker defined rules/chains; otherwise, our docker ports are exposed to the world.

In our current state, we have some rules that are simply not yet migrated and others that perform marking, logging, and TEE'ing. We also have chains set aside for automation in our provisioning process that allow us to poke holes in the firewall for machines that are being provisioned but are not yet added as nodes; and, in fact, the IPs used for provisioning are completely different from those in production. We could have a calico policy allowing all possible provisioning prefixes, but we don't want things that open because it would be a large number of prefixes as they are tired to our edge locations. We're also reluctant to grant access and have the service manipulate calico policies, via etcd or otherwise.