search

submariner-io / subctl

CLI tool that provides helper commands to install, inspect, and troubleshoot a Submariner deployment.
Apache License 2.0
18 stars 22 forks source link

Use iptables-save instead of explicit lists of tables #1205

Open skitt opened 1 month ago

skitt commented 1 month ago

This was inspired by the recent addition of the mangle table in iptables output for subctl gather. Instead of manually specifying which tables to log, it uses iptables-save to automatically determine all the active tables.

The output is not as legible so this may not be appropriate. For comparison, here’s the current output from iptables -L for the nat table:

iptables -L -n -v --line-numbers -t nat
Chain PREROUTING (policy ACCEPT 27 packets, 2597 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1       27  2590 KUBE-SERVICES  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
2        0     0 DOCKER_OUTPUT  0    --  *      *       0.0.0.0/0            172.18.0.1          

Chain INPUT (policy ACCEPT 20 packets, 2149 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 143 packets, 8993 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1      368 27679 KUBE-SERVICES  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
2       90  6450 DOCKER_OUTPUT  0    --  *      *       0.0.0.0/0            172.18.0.1          

Chain POSTROUTING (policy ACCEPT 135 packets, 9150 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1      198 16020 SUBMARINER-POSTROUTING  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
2      371 27867 KUBE-POSTROUTING  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes postrouting rules */
3        0     0 DOCKER_POSTROUTING  0    --  *      *       0.0.0.0/0            172.18.0.1          
4      101  6481 KIND-MASQ-AGENT  0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type !LOCAL /* kind-masq-agent: ensure nat POSTROUTING directs all non-LOCAL destination traffic to our custom KIND-MASQ-AGENT chain */

Chain DOCKER_OUTPUT (2 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DNAT       6    --  *      *       0.0.0.0/0            172.18.0.1           tcp dpt:53 to:127.0.0.11:38321
2       90  6450 DNAT       17   --  *      *       0.0.0.0/0            172.18.0.1           udp dpt:53 to:127.0.0.11:38892

Chain DOCKER_POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 SNAT       6    --  *      *       127.0.0.11           0.0.0.0/0            tcp spt:38321 to:172.18.0.1:53
2        0     0 SNAT       17   --  *      *       127.0.0.11           0.0.0.0/0            udp spt:38892 to:172.18.0.1:53

Chain KIND-MASQ-AGENT (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 RETURN     0    --  *      *       0.0.0.0/0            10.131.0.0/16        /* kind-masq-agent: local traffic is not subject to MASQUERADE */
2      101  6481 MASQUERADE  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kind-masq-agent: outbound traffic is subject to MASQUERADE (must be last in chain) */

Chain KUBE-KUBELET-CANARY (0 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain KUBE-MARK-MASQ (15 references)
num   pkts bytes target     prot opt in     out     source               destination         
1      169 14288 MARK       0    --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000

Chain KUBE-NODEPORTS (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain KUBE-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1       30  1800 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            mark match ! 0x4000/0x4000
2      169 14288 MARK       0    --  *      *       0.0.0.0/0            0.0.0.0/0            MARK xor 0x4000
3      169 14288 MASQUERADE  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service traffic requiring SNAT */ random-fully

Chain KUBE-PROXY-CANARY (0 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain KUBE-SEP-6MTZSWVZ7DKAKVC6 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       172.18.0.7           0.0.0.0/0            /* default/kubernetes:https */
2        4   240 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ tcp to:172.18.0.7:6443

Chain KUBE-SEP-CCIDRBOGP46NCIZT (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.0.4           0.0.0.0/0            /* kube-system/kube-dns:metrics */
2        0     0 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics */ tcp to:10.131.0.4:9153

Chain KUBE-SEP-EHRALZ2IVFLIEMXK (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.0.2           0.0.0.0/0            /* kube-system/kube-dns:dns */
2      102  8638 DNAT       17   --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:10.131.0.2:53

Chain KUBE-SEP-EWDMSLOSUEG5PNZG (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.0.4           0.0.0.0/0            /* kube-system/kube-dns:dns */
2       90  7626 DNAT       17   --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:10.131.0.4:53

Chain KUBE-SEP-FX75HGKHH5NVZOM4 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.1.3           0.0.0.0/0            /* submariner-operator/submariner-operator-metrics:metrics */
2        0     0 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* submariner-operator/submariner-operator-metrics:metrics */ tcp to:10.131.1.3:8383

Chain KUBE-SEP-O474JOZSQBGFL7ND (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.1.4           0.0.0.0/0            /* submariner-operator/submariner-gateway-metrics:metrics */
2        0     0 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* submariner-operator/submariner-gateway-metrics:metrics */ tcp to:10.131.1.4:8080

Chain KUBE-SEP-T7O47GHNBAIDBEIK (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.0.4           0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
2        0     0 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:10.131.0.4:53

Chain KUBE-SEP-UDNZKJBLV34OAAWT (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.0.2           0.0.0.0/0            /* kube-system/kube-dns:metrics */
2        0     0 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics */ tcp to:10.131.0.2:9153

Chain KUBE-SEP-YWARMXSG3QHBE23Z (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  0    --  *      *       10.131.0.2           0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
2        0     0 DNAT       6    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:10.131.0.2:53

Chain KUBE-SERVICES (2 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-SVC-JD5MR3NA4I4DYORP  6    --  *      *       0.0.0.0/0            100.67.0.10          /* kube-system/kube-dns:metrics cluster IP */ tcp dpt:9153
2        0     0 KUBE-SVC-BHLC6BRLQPHUI46H  6    --  *      *       0.0.0.0/0            100.67.9.143         /* submariner-operator/submariner-operator-metrics:metrics cluster IP */ tcp dpt:8383
3        0     0 KUBE-SVC-5YA5POTRUUKRWHKC  6    --  *      *       0.0.0.0/0            100.67.19.18         /* submariner-operator/submariner-gateway-metrics:metrics cluster IP */ tcp dpt:8080
4        1    60 KUBE-SVC-NPX46M4PTMTKRN6Y  6    --  *      *       0.0.0.0/0            100.67.0.1           /* default/kubernetes:https cluster IP */ tcp dpt:443
5      168 14228 KUBE-SVC-TCOU7JCQXEZGVUNU  17   --  *      *       0.0.0.0/0            100.67.0.10          /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
6        0     0 KUBE-SVC-ERIFXISQEP7F7OF4  6    --  *      *       0.0.0.0/0            100.67.0.10          /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
7       33  2020 KUBE-NODEPORTS  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL

Chain KUBE-SVC-5YA5POTRUUKRWHKC (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  6    --  *      *      !10.131.0.0/16        100.67.19.18         /* submariner-operator/submariner-gateway-metrics:metrics cluster IP */ tcp dpt:8080
2        0     0 KUBE-SEP-O474JOZSQBGFL7ND  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* submariner-operator/submariner-gateway-metrics:metrics -> 10.131.1.4:8080 */

Chain KUBE-SVC-BHLC6BRLQPHUI46H (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  6    --  *      *      !10.131.0.0/16        100.67.9.143         /* submariner-operator/submariner-operator-metrics:metrics cluster IP */ tcp dpt:8383
2        0     0 KUBE-SEP-FX75HGKHH5NVZOM4  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* submariner-operator/submariner-operator-metrics:metrics -> 10.131.1.3:8383 */

Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  6    --  *      *      !10.131.0.0/16        100.67.0.10          /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
2        0     0 KUBE-SEP-YWARMXSG3QHBE23Z  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp -> 10.131.0.2:53 */ statistic mode random probability 0.50000000000
3        0     0 KUBE-SEP-T7O47GHNBAIDBEIK  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp -> 10.131.0.4:53 */

Chain KUBE-SVC-JD5MR3NA4I4DYORP (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 KUBE-MARK-MASQ  6    --  *      *      !10.131.0.0/16        100.67.0.10          /* kube-system/kube-dns:metrics cluster IP */ tcp dpt:9153
2        0     0 KUBE-SEP-UDNZKJBLV34OAAWT  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics -> 10.131.0.2:9153 */ statistic mode random probability 0.50000000000
3        0     0 KUBE-SEP-CCIDRBOGP46NCIZT  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics -> 10.131.0.4:9153 */

Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        3   180 KUBE-MARK-MASQ  6    --  *      *      !10.131.0.0/16        100.67.0.1           /* default/kubernetes:https cluster IP */ tcp dpt:443
2        4   240 KUBE-SEP-6MTZSWVZ7DKAKVC6  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https -> 172.18.0.7:6443 */

Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1      192 16264 KUBE-MARK-MASQ  17   --  *      *      !10.131.0.0/16        100.67.0.10          /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
2      102  8638 KUBE-SEP-EHRALZ2IVFLIEMXK  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns -> 10.131.0.2:53 */ statistic mode random probability 0.50000000000
3       90  7626 KUBE-SEP-EWDMSLOSUEG5PNZG  0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns -> 10.131.0.4:53 */

Chain SUBMARINER-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 SNAT       0    --  *      vx-submariner  240.0.0.0/8          0.0.0.0/0            to:10.131.1.1
2        0     0 ACCEPT     0    --  *      *       10.131.0.0/16        100.66.0.0/16       
3        0     0 ACCEPT     0    --  *      *       100.66.0.0/16        10.131.0.0/16       
4        1    52 ACCEPT     0    --  *      *       10.131.0.0/16        10.130.0.0/16       
5        0     0 ACCEPT     0    --  *      *       10.130.0.0/16        10.131.0.0/16

and here’s the corresponding output from iptables-save -c:

# Generated by iptables-save v1.8.9 (nf_tables) on Thu Aug  8 14:03:30 2024
*nat
:PREROUTING ACCEPT [26:2537]
:INPUT ACCEPT [19:2089]
:OUTPUT ACCEPT [142:8933]
:POSTROUTING ACCEPT [134:9090]
:DOCKER_OUTPUT - [0:0]
:DOCKER_POSTROUTING - [0:0]
:KIND-MASQ-AGENT - [0:0]
:KUBE-KUBELET-CANARY - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-PROXY-CANARY - [0:0]
:KUBE-SEP-6MTZSWVZ7DKAKVC6 - [0:0]
:KUBE-SEP-CCIDRBOGP46NCIZT - [0:0]
:KUBE-SEP-EHRALZ2IVFLIEMXK - [0:0]
:KUBE-SEP-EWDMSLOSUEG5PNZG - [0:0]
:KUBE-SEP-FX75HGKHH5NVZOM4 - [0:0]
:KUBE-SEP-O474JOZSQBGFL7ND - [0:0]
:KUBE-SEP-T7O47GHNBAIDBEIK - [0:0]
:KUBE-SEP-UDNZKJBLV34OAAWT - [0:0]
:KUBE-SEP-YWARMXSG3QHBE23Z - [0:0]
:KUBE-SERVICES - [0:0]
:KUBE-SVC-5YA5POTRUUKRWHKC - [0:0]
:KUBE-SVC-BHLC6BRLQPHUI46H - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-JD5MR3NA4I4DYORP - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:SUBMARINER-POSTROUTING - [0:0]
[26:2530] -A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
[0:0] -A PREROUTING -d 172.18.0.1/32 -j DOCKER_OUTPUT
[367:27619] -A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
[90:6450] -A OUTPUT -d 172.18.0.1/32 -j DOCKER_OUTPUT
[197:15960] -A POSTROUTING -j SUBMARINER-POSTROUTING
[370:27807] -A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
[0:0] -A POSTROUTING -d 172.18.0.1/32 -j DOCKER_POSTROUTING
[101:6481] -A POSTROUTING -m addrtype ! --dst-type LOCAL -m comment --comment "kind-masq-agent: ensure nat POSTROUTING directs all non-LOCAL destination traffic to our custom KIND-MASQ-AGENT chain" -j KIND-MASQ-AGENT
[0:0] -A DOCKER_OUTPUT -d 172.18.0.1/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 127.0.0.11:38321
[90:6450] -A DOCKER_OUTPUT -d 172.18.0.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination 127.0.0.11:38892
[0:0] -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p tcp -m tcp --sport 38321 -j SNAT --to-source 172.18.0.1:53
[0:0] -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p udp -m udp --sport 38892 -j SNAT --to-source 172.18.0.1:53
[0:0] -A KIND-MASQ-AGENT -d 10.131.0.0/16 -m comment --comment "kind-masq-agent: local traffic is not subject to MASQUERADE" -j RETURN
[101:6481] -A KIND-MASQ-AGENT -m comment --comment "kind-masq-agent: outbound traffic is subject to MASQUERADE (must be last in chain)" -j MASQUERADE
[169:14288] -A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
[29:1740] -A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
[169:14288] -A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
[169:14288] -A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE --random-fully
[0:0] -A KUBE-SEP-6MTZSWVZ7DKAKVC6 -s 172.18.0.7/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
[4:240] -A KUBE-SEP-6MTZSWVZ7DKAKVC6 -p tcp -m comment --comment "default/kubernetes:https" -m tcp -j DNAT --to-destination 172.18.0.7:6443
[0:0] -A KUBE-SEP-CCIDRBOGP46NCIZT -s 10.131.0.4/32 -m comment --comment "kube-system/kube-dns:metrics" -j KUBE-MARK-MASQ
[0:0] -A KUBE-SEP-CCIDRBOGP46NCIZT -p tcp -m comment --comment "kube-system/kube-dns:metrics" -m tcp -j DNAT --to-destination 10.131.0.4:9153
[0:0] -A KUBE-SEP-EHRALZ2IVFLIEMXK -s 10.131.0.2/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
[102:8638] -A KUBE-SEP-EHRALZ2IVFLIEMXK -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.131.0.2:53
[0:0] -A KUBE-SEP-EWDMSLOSUEG5PNZG -s 10.131.0.4/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
[90:7626] -A KUBE-SEP-EWDMSLOSUEG5PNZG -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.131.0.4:53
[0:0] -A KUBE-SEP-FX75HGKHH5NVZOM4 -s 10.131.1.3/32 -m comment --comment "submariner-operator/submariner-operator-metrics:metrics" -j KUBE-MARK-MASQ
[0:0] -A KUBE-SEP-FX75HGKHH5NVZOM4 -p tcp -m comment --comment "submariner-operator/submariner-operator-metrics:metrics" -m tcp -j DNAT --to-destination 10.131.1.3:8383
[0:0] -A KUBE-SEP-O474JOZSQBGFL7ND -s 10.131.1.4/32 -m comment --comment "submariner-operator/submariner-gateway-metrics:metrics" -j KUBE-MARK-MASQ
[0:0] -A KUBE-SEP-O474JOZSQBGFL7ND -p tcp -m comment --comment "submariner-operator/submariner-gateway-metrics:metrics" -m tcp -j DNAT --to-destination 10.131.1.4:8080
[0:0] -A KUBE-SEP-T7O47GHNBAIDBEIK -s 10.131.0.4/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
[0:0] -A KUBE-SEP-T7O47GHNBAIDBEIK -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.131.0.4:53
[0:0] -A KUBE-SEP-UDNZKJBLV34OAAWT -s 10.131.0.2/32 -m comment --comment "kube-system/kube-dns:metrics" -j KUBE-MARK-MASQ
[0:0] -A KUBE-SEP-UDNZKJBLV34OAAWT -p tcp -m comment --comment "kube-system/kube-dns:metrics" -m tcp -j DNAT --to-destination 10.131.0.2:9153
[0:0] -A KUBE-SEP-YWARMXSG3QHBE23Z -s 10.131.0.2/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
[0:0] -A KUBE-SEP-YWARMXSG3QHBE23Z -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.131.0.2:53
[0:0] -A KUBE-SERVICES -d 100.67.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:metrics cluster IP" -m tcp --dport 9153 -j KUBE-SVC-JD5MR3NA4I4DYORP
[0:0] -A KUBE-SERVICES -d 100.67.9.143/32 -p tcp -m comment --comment "submariner-operator/submariner-operator-metrics:metrics cluster IP" -m tcp --dport 8383 -j KUBE-SVC-BHLC6BRLQPHUI46H
[0:0] -A KUBE-SERVICES -d 100.67.19.18/32 -p tcp -m comment --comment "submariner-operator/submariner-gateway-metrics:metrics cluster IP" -m tcp --dport 8080 -j KUBE-SVC-5YA5POTRUUKRWHKC
[1:60] -A KUBE-SERVICES -d 100.67.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
[168:14228] -A KUBE-SERVICES -d 100.67.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
[0:0] -A KUBE-SERVICES -d 100.67.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
[31:1900] -A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
[0:0] -A KUBE-SVC-5YA5POTRUUKRWHKC ! -s 10.131.0.0/16 -d 100.67.19.18/32 -p tcp -m comment --comment "submariner-operator/submariner-gateway-metrics:metrics cluster IP" -m tcp --dport 8080 -j KUBE-MARK-MASQ
[0:0] -A KUBE-SVC-5YA5POTRUUKRWHKC -m comment --comment "submariner-operator/submariner-gateway-metrics:metrics -> 10.131.1.4:8080" -j KUBE-SEP-O474JOZSQBGFL7ND
[0:0] -A KUBE-SVC-BHLC6BRLQPHUI46H ! -s 10.131.0.0/16 -d 100.67.9.143/32 -p tcp -m comment --comment "submariner-operator/submariner-operator-metrics:metrics cluster IP" -m tcp --dport 8383 -j KUBE-MARK-MASQ
[0:0] -A KUBE-SVC-BHLC6BRLQPHUI46H -m comment --comment "submariner-operator/submariner-operator-metrics:metrics -> 10.131.1.3:8383" -j KUBE-SEP-FX75HGKHH5NVZOM4
[0:0] -A KUBE-SVC-ERIFXISQEP7F7OF4 ! -s 10.131.0.0/16 -d 100.67.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
[0:0] -A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.131.0.2:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-YWARMXSG3QHBE23Z
[0:0] -A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.131.0.4:53" -j KUBE-SEP-T7O47GHNBAIDBEIK
[0:0] -A KUBE-SVC-JD5MR3NA4I4DYORP ! -s 10.131.0.0/16 -d 100.67.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:metrics cluster IP" -m tcp --dport 9153 -j KUBE-MARK-MASQ
[0:0] -A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment "kube-system/kube-dns:metrics -> 10.131.0.2:9153" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-UDNZKJBLV34OAAWT
[0:0] -A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment "kube-system/kube-dns:metrics -> 10.131.0.4:9153" -j KUBE-SEP-CCIDRBOGP46NCIZT
[3:180] -A KUBE-SVC-NPX46M4PTMTKRN6Y ! -s 10.131.0.0/16 -d 100.67.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
[4:240] -A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https -> 172.18.0.7:6443" -j KUBE-SEP-6MTZSWVZ7DKAKVC6
[192:16264] -A KUBE-SVC-TCOU7JCQXEZGVUNU ! -s 10.131.0.0/16 -d 100.67.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
[102:8638] -A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.131.0.2:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-EHRALZ2IVFLIEMXK
[90:7626] -A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.131.0.4:53" -j KUBE-SEP-EWDMSLOSUEG5PNZG
[0:0] -A SUBMARINER-POSTROUTING -s 240.0.0.0/8 -o vx-submariner -j SNAT --to-source 10.131.1.1
[0:0] -A SUBMARINER-POSTROUTING -s 10.131.0.0/16 -d 100.66.0.0/16 -j ACCEPT
[0:0] -A SUBMARINER-POSTROUTING -s 100.66.0.0/16 -d 10.131.0.0/16 -j ACCEPT
[1:52] -A SUBMARINER-POSTROUTING -s 10.131.0.0/16 -d 10.130.0.0/16 -j ACCEPT
[0:0] -A SUBMARINER-POSTROUTING -s 10.130.0.0/16 -d 10.131.0.0/16 -j ACCEPT
COMMIT
# Completed on Thu Aug  8 14:03:30 2024
submariner-bot commented 1 month ago

🤖 Created branch: z_pr1205/skitt/use-iptables-save 🚀 Full E2E won't run until the "ready-to-test" label is applied. I will add it automatically once the PR has 2 approvals, or you can add it manually.

yboaron commented 1 month ago

I think syntax [1] is more convenient for debugging/troubleshooting than [2].

[1]

Chain SUBMARINER-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 SNAT       0    --  *      vx-submariner  240.0.0.0/8          0.0.0.0/0            to:10.131.1.1
2        0     0 ACCEPT     0    --  *      *       10.131.0.0/16        100.66.0.0/16       
3        0     0 ACCEPT     0    --  *      *       100.66.0.0/16        10.131.0.0/16       
4        1    52 ACCEPT     0    --  *      *       10.131.0.0/16        10.130.0.0/16       
5        0     0 ACCEPT     0    --  *      *       10.130.0.0/16        10.131.0.0/16

[2] 

[0:0] -A SUBMARINER-POSTROUTING -s 240.0.0.0/8 -o vx-submariner -j SNAT --to-source 10.131.1.1
[0:0] -A SUBMARINER-POSTROUTING -s 10.131.0.0/16 -d 100.66.0.0/16 -j ACCEPT
[0:0] -A SUBMARINER-POSTROUTING -s 100.66.0.0/16 -d 10.131.0.0/16 -j ACCEPT
[1:52] -A SUBMARINER-POSTROUTING -s 10.131.0.0/16 -d 10.130.0.0/16 -j ACCEPT
[0:0] -A SUBMARINER-POSTROUTING -s 10.130.0.0/16 -d 10.131.0.0/16 -j ACCEPT
vthapar commented 1 month ago

What do these numbers represent, pkts/bytes?

:PREROUTING ACCEPT [26:2537]

[1] one does look more legible but that could also be due to familiarity. What I like about [2] is that it looks like we could copy/paste it to apply those rules. It may come in handy if we tomorrow explore option to recreate/simulate setup from gather output. We had done something similar with OVS Openflow rules though not sure if it would be practical in this case as OVS by design lends well to it unlike IP Tables.

skitt commented 1 month ago

What do these numbers represent, pkts/bytes?

:PREROUTING ACCEPT [26:2537]

Yes, packets:bytes — you can see similar values in the iptables output (27 packets, 2590 bytes).

github-actions[bot] commented 2 weeks ago

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 5 days ago

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.