[Feature Request] Support additional actions such as `gotopriority` and `continue`

[Dyanngg]

In Antrea we use OVS as the datapath for realizing NetworkPolicy for Kubernetes networking. In most cases, packet matching for NetworkPolicies can be efficiently expressed in OpenFlow as a conjunctive match on three dimensions, which is nw_src, nw_dst and tcp/udp/icmp_dst. In Antrea, users can create such policies at different priorities, and for each of these policy rules we translate to conjunctive action flows at different priorities in a particular table (for scalability reasons, since there can be a large number of unique priorities for these policies). For example:

table=45, priority=14000,conj_id=1,ip actions=drop                   <- policy rule 1
table=45, priority=14000,ip,nw_src=10.10.1.6 actions=conjunction(1,1/3)
table=45, priority=14000,ip,nw_dst=10.10.1.8 actions=conjunction(1,2/3)
table=45, priority=14000,udp,tp_dst=53 actions=conjunction(1,3/3)
table=45, priority=13500,conj_id=2,ip actions=resubmit(,50)          <- policy rule 2
table=45, priority=13500,ip,nw_src=10.10.1.6 actions=conjunction(2,1/3)
table=45, priority=13500,ip,nw_dst=10.10.1.9 actions=conjunction(2,2/3)
table=45, priority=13500,tcp,tp_dst=53 actions=conjunction(2,3/3)
..........

However, recently we come across a usecase where NetworkPolicy in K8s might be defined as, "drop any traffic to 10.0.0.3 if not coming from 10.0.0.4 or 10.0.0.5; If the packet is coming from these 'exception' addresses, continue to evaluate for lower priority NetworkPolicy rules" The 'exception's can be individual addresses or a CIDR, such as 10.0.0.1/24. This has become a problem when we try to translate to match conditions in OVS: openFlow matching does not support negation match expressions such as nw_src!=10.0.0.4.

Following the suggestions of inequality match (https://man7.org/linux/man-pages/man7/ovs-fields.7.html), the best solution we could think of is to compute the IP address complement of 10.0.0.4 and 10.0.0.5, in CIDR format, and use these explicit networks as packet matching criteria for nw_src or nw_dst. However, the above example alone would yield 31 CIDRs, and 1. ipv6 would make the problem even worse (excluding a single address in ipv6 address space will lead to 128 different CIDRs) 2. excluding more than one address can potentially add to the number of CIDRs that expresses the address set complement.

As an alternative, we think the OVS flows would be more clean, readable and scalable if there's additional actions supported such as gotopriority(if the packet is matched, continue to match other flows in the table starting from a lower than current priority) or continue (if the packet is matched, continue to go thru the openflow pipeline from the point of match). How it would look like for the example above:

table=45, priority=14000,conj_id=1,ip actions=gotopriority:13000       <- policy rule, skip certain addresses
table=45, priority=14000,ip,nw_src=10.10.1.4 actions=conjunction(1,1/2)
table=45, priority=14000,ip,nw_src=10.10.1.5 actions=conjunction(1,1/2)
table=45, priority=14000,ip,nw_dst=10.10.1.3 actions=conjunction(1,2/2)
table=45, priority=13500,ip,nw_dst=10.10.1.3 actions=drop              <- policy rule, drop otherwise
table=45, priority=13000, ..............                        <- other lower priority policy rules

Based on our preliminary discussions with @williamtu, it seems that the functionality can be achieved with reasonable dev effort. We believe these actions can bring additional flexibility in OVS in terms of negation matching/skip certain flows if some conditions are matched, in general.

/cc @williamtu @wenyingd @GraysonWu @qiyueyao

[igsilya]

Hi @Dyanngg , what would be the difference between your suggestion and something like this:

table=45, priority=99999,ip actions=set_field:14000->reg1,resubmit(,45)
table=45, priority=14000,reg1=14000,conj_id=1,ip actions=set_field:13000->reg1,resubmit(,45)
table=45, priority=14000,reg1=14000,ip,nw_src=10.10.1.4 actions=conjunction(1,1/2)
table=45, priority=14000,reg1=14000,ip,nw_src=10.10.1.5 actions=conjunction(1,1/2)
table=45, priority=14000,reg1=14000,ip,nw_dst=10.10.1.3 actions=conjunction(1,2/2)
table=45, priority=13500,reg1=14000,ip actions=drop
table=45, priority=13000,reg1=13000, ..............

Shouldn't it be the same? I might misspell some syntax, but I hope that it illustrates the idea. (I dropped a match on nw_dst in the drop flow, it seemed incorrect, isn't it?)