search

spidernet-io / egressgateway

Network egress policy for Kubernetes
https://spidernet-io.github.io/egressgateway/
Apache License 2.0
212 stars 16 forks source link

software bug #1536

Open amirdaaee opened 3 weeks ago

amirdaaee commented 3 weeks ago

Describe the version v0.6.0

Describe the bug can not create policy on ipv6 gateways

How To Reproduce

gateway manifest:

apiVersion: egressgateway.spidernet.io/v1beta1
kind: EgressGateway
metadata:
  name: egress-v6
spec:
  ippools:
    ipv6:
      - "19a4:7b80:409f:ad95:e6aa:a566:0cc9:1-19a4:7b80:409f:ad95:e6aa:a566:0cc9:10"
    ipv6DefaultEIP: "19a4:7b80:409f:ad95:e6aa:a566:0cc9:1"
  nodeSelector:
    selector:
      matchLabels:
        ...

policy manifest:

apiVersion: egressgateway.spidernet.io/v1beta1
kind: EgressPolicy
metadata:
  name: policy-v6
  namespace: default
spec:
  egressGatewayName: egress-v6
  egressIP:
    ipv6: "19a4:7b80:409f:ad95:e6aa:a566:0cc9:10"
  appliedTo:
    podSelector:
      matchLabels:
        app: "visitor"

Expected behavior expect it to work

Screenshots and log following error logs are available on agent pod:

{"level":"info","ts":"2024-11-06T15:16:23.338Z","caller":"iptables/table.go:1107","msg":"failed to execute ip(6)tables-restore command","table":"nat","ipVersion":4,"output":"","errorOutput":"iptables-nft-restore v1.8.7 (nf_tables): option \"--to-source\" requires an argument\nError occurred at line: 3\nTry `iptables-nft-restore -h' or 'iptables-nft-restore --help' for more information.\n","warn":"exit status 2","input":"*nat\n:EGRESSGATEWAY-SNAT-EIP - -\n-A EGRESSGATEWAY-SNAT-EIP -m comment --comment \"egw:stiH8C7LO4XD9Cqm\" -m comment --comment \"snat policy default-test-6\" -m set --match-set egress-src-v4-b3127420712c25970 src -m set ! --match-set egress-cluster-cidr-ipv4 dst -m conntrack --ctdir ORIGINAL --jump SNAT --to-source \n-I PREROUTING -m comment --comment \"egw:bx14gEwdI2z4JR7V\" -m comment --comment \"EgressGateway traffic accept datapath rule\" -m mark --mark 0x26000000/0xff000000 --jump ACCEPT\nCOMMIT\n"}

{"level":"error","ts":"2024-11-06T15:16:23.338Z","caller":"iptables/table.go:805","msg":"failed to program iptables, loading diags before panic.","table":"nat","ipVersion":4,"error":"exit status 2","stacktrace":"github.com/spidernet-io/egressgateway/pkg/iptables.(Table).Apply\n\t/src/pkg/iptables/table.go:805\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).initApplyPolicy\n\t/src/pkg/agent/police.go:304\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).Reconcile.func1\n\t/src/pkg/agent/police.go:65\nsync.(Once).doSlow\n\t/usr/local/go/src/sync/once.go:74\nsync.(Once).Do\n\t/usr/local/go/src/sync/once.go:65\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).Reconcile\n\t/src/pkg/agent/police.go:62\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222"}

{"level":"error","ts":"2024-11-06T15:16:23.352Z","caller":"iptables/table.go:811","msg":"","table":"nat","ipVersion":4,"iptablesState":"# Generated by iptables-nft-save v1.8.7 on Wed Nov 6 15:16:23 2024\n*nat\n:PREROUTING ACCEPT [0:0]\n:INPUT ACCEPT [0:0]\n:OUTPUT ACCEPT [0:0]\n:POSTROUTING ACCEPT [0:0]\n:EGRESSGATEWAY-SNAT-EIP - [0:0]\n:KUBE-EXT-2OFMA5BBWKIWDPSD - [0:0]\n:KUBE-EXT-7U7O5RDNYVDOUGBF - [0:0]\n:KUBE-EXT-NHXMRZ4FDHP77GE3 - [0:0]\n:KUBE-KUBELET-CANARY - [0:0]\n:KUBE-MARK-MASQ - [0:0]\n:KUBE-NODEPORTS - [0:0]\n:KUBE-POSTROUTING - [0:0]\n:KUBE-PROXY-CANARY - [0:0]\n:KUBE-SEP-34ERLKPYC6DKDBGS - [0:0]\n:KUBE-SEP-4HXEATJ23FYNZIVK - [0:0]\n:KUBE-SEP-66QAXE6EYTRCMRYX - [0:0]\n:KUBE-SEP-A6KMR36ACIBYXLOJ - [0:0]\n:KUBE-SEP-BG6TFBBQGAWE3Z7L - [0:0]\n:KUBE-SEP-EPZMDUCJDC2OHYAK - [0:0]\n:KUBE-SEP-I4ZNX4OY6UO2PD2U - [0:0]\n:KUBE-SEP-IXUGJRA4GZQLZAJU - [0:0]\n:KUBE-SEP-JB5LSQZZDXLTL7QB - [0:0]\n:KUBE-SEP-OEESFSBXALRV4VWG - [0:0]\n:KUBE-SEP-RTIEMOUJKRROVH3J - [0:0]\n:KUBE-SEP-SE3MTBCZ3RW6IAMO - [0:0]\n:KUBE-SEP-TJ33L5TODYNVESW3 - [0:0]\n:KUBE-SEP-U2LOLC3TDNOYZEYN - [0:0]\n:KUBE-SEP-UZI23UX3WZZ3NTLJ - [0:0]\n:KUBE-SEP-VO2WZYMO5RK474PI - [0:0]\n:KUBE-SEP-WDTIMUUGW3CTM56Y - [0:0]\n:KUBE-SEP-X6NJHHYNJCAAN324 - [0:0]\n:KUBE-SEP-YFS4TYSYNIK72KIF - [0:0]\n:KUBE-SERVICES - [0:0]\n:KUBE-SVC-2OFMA5BBWKIWDPSD - [0:0]\n:KUBE-SVC-7CXIK4VIXYOGPYZ7 - [0:0]\n:KUBE-SVC-7U7O5RDNYVDOUGBF - [0:0]\n:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]\n:KUBE-SVC-FTKCEC5VWOO5DYBC - [0:0]\n:KUBE-SVC-GZ25SP4UFGF7SAVL - [0:0]\n:KUBE-SVC-I24EZXP75AX5E7TU - [0:0]\n:KUBE-SVC-JD5MR3NA4I4DYORP - [0:0]\n:KUBE-SVC-M3XNSA5ZTED24LO2 - [0:0]\n:KUBE-SVC-MCYIF6SJCIAJCF25 - [0:0]\n:KUBE-SVC-NHXMRZ4FDHP77GE3 - [0:0]\n:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]\n:KUBE-SVC-RK657RLKDNVNU64O - [0:0]\n:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]\n:KUBE-SVC-UNMNSATZFYJDMRLS - [0:0]\n:KUBE-SVL-2OFMA5BBWKIWDPSD - [0:0]\n:KUBE-SVL-7U7O5RDNYVDOUGBF - [0:0]\n:KUBE-SVL-NHXMRZ4FDHP77GE3 - [0:0]\n:cali-OUTPUT - [0:0]\n:cali-POSTROUTING - [0:0]\n:cali-PREROUTING - [0:0]\n:cali-fip-dnat - [0:0]\n:cali-fip-snat - [0:0]\n:cali-nat-outgoing - [0:0]\n-A PREROUTING -m comment --comment \"kubernetes service portals\" -j KUBE-SERVICES\n-A PREROUTING -m comment --comment \"cali:6gwbT8clXdHdC1b1\" -j cali-PREROUTING\n-A OUTPUT -m comment --comment \"kubernetes service portals\" -j KUBE-SERVICES\n-A OUTPUT -m comment --comment \"cali:tVnHkvAo15HuiPy0\" -j cali-OUTPUT\n-A POSTROUTING -m comment --comment \"egw:x1tdBi75jif7GCxh\" -m comment --comment \"SNAT for egress traffic\" -j EGRESSGATEWAY-SNAT-EIP\n-A POSTROUTING -m comment --comment \"egw:lefvkAAcigCbsdOb\" -m comment --comment \"Accept for egress traffic from pod going to EgressTunnel\" -m mark --mark 0x26000000 -j ACCEPT\n-A POSTROUTING -m comment --comment \"kubernetes postrouting rules\" -j KUBE-POSTROUTING\n-A POSTROUTING -m comment --comment \"cali:0i8pjzKKPyA34aQD\" -j cali-POSTROUTING\n-A KUBE-EXT-2OFMA5BBWKIWDPSD -s 10.244.0.0/16 -m comment --comment \"pod traffic for newmyservice/myservice-newshcan-envoy-env1:http external destinations\" -j KUBE-SVC-2OFMA5BBWKIWDPSD\n-A KUBE-EXT-2OFMA5BBWKIWDPSD -m comment --comment \"masquerade LOCAL traffic for newmyservice/myservice-newshcan-envoy-env1:http external destinations\" -m addrtype --src-type LOCAL -j KUBE-MARK-MASQ\n-A KUBE-EXT-2OFMA5BBWKIWDPSD -m comment --comment \"route LOCAL traffic for newmyservice/myservice-newshcan-envoy-env1:http external destinations\" -m addrtype --src-type LOCAL -j KUBE-SVC-2OFMA5BBWKIWDPSD\n-A KUBE-EXT-2OFMA5BBWKIWDPSD -j KUBE-SVL-2OFMA5BBWKIWDPSD\n-A KUBE-EXT-7U7O5RDNYVDOUGBF -s 10.244.0.0/16 -m comment --comment \"pod traffic for newmyservice/myservice-newshcan-envoy-env1:tls external destinations\" -j KUBE-SVC-7U7O5RDNYVDOUGBF\n-A KUBE-EXT-7U7O5RDNYVDOUGBF -m comment --comment \"masquerade LOCAL traffic for newmyservice/myservice-newshcan-envoy-env1:tls external destinations\" -m addrtype --src-type LOCAL -j KUBE-MARK-MASQ\n-A KUBE-EXT-7U7O5RDNYVDOUGBF -m comment --comment \"route LOCAL traffic for newmyservice/myservice-newshcan-envoy-env1:tls external destinations\" -m addrtype --src-type LOCAL -j KUBE-SVC-7U7O5RDNYVDOUGBF\n-A KUBE-EXT-7U7O5RDNYVDOUGBF -j KUBE-SVL-7U7O5RDNYVDOUGBF\n-A KUBE-EXT-NHXMRZ4FDHP77GE3 -s 10.244.0.0/16 -m comment --comment \"pod traffic for newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp external destinations\" -j KUBE-SVC-NHXMRZ4FDHP77GE3\n-A KUBE-EXT-NHXMRZ4FDHP77GE3 -m comment --comment \"masquerade LOCAL traffic for newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp external destinations\" -m addrtype --src-type LOCAL -j KUBE-MARK-MASQ\n-A KUBE-EXT-NHXMRZ4FDHP77GE3 -m comment --comment \"route LOCAL traffic for newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp external destinations\" -m addrtype --src-type LOCAL -j KUBE-SVC-NHXMRZ4FDHP77GE3\n-A KUBE-EXT-NHXMRZ4FDHP77GE3 -j KUBE-SVL-NHXMRZ4FDHP77GE3\n-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000\n-A KUBE-NODEPORTS -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls\" -j KUBE-EXT-7U7O5RDNYVDOUGBF\n-A KUBE-NODEPORTS -p udp -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp\" -j KUBE-EXT-NHXMRZ4FDHP77GE3\n-A KUBE-NODEPORTS -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http\" -j KUBE-EXT-2OFMA5BBWKIWDPSD\n-A KUBE-POSTROUTING -j RETURN\n-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0\n-A KUBE-POSTROUTING -m comment --comment \"kubernetes service traffic requiring SNAT\" -j MASQUERADE --random-fully\n-A KUBE-SEP-34ERLKPYC6DKDBGS -s 10.244.109.2/32 -m comment --comment \"kube-system/kube-dns:metrics\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-34ERLKPYC6DKDBGS -p tcp -m comment --comment \"kube-system/kube-dns:metrics\" -m tcp -j DNAT --to-destination 10.244.109.2:9153\n-A KUBE-SEP-4HXEATJ23FYNZIVK -s 10.244.109.13/32 -m comment --comment \"metallb-system/metallb-webhook-service\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-4HXEATJ23FYNZIVK -p tcp -m comment --comment \"metallb-system/metallb-webhook-service\" -m tcp -j DNAT --to-destination 10.244.109.13:9443\n-A KUBE-SEP-66QAXE6EYTRCMRYX -s 10.244.109.2/32 -m comment --comment \"kube-system/kube-dns:dns-tcp\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-66QAXE6EYTRCMRYX -p tcp -m comment --comment \"kube-system/kube-dns:dns-tcp\" -m tcp -j DNAT --to-destination 10.244.109.2:53\n-A KUBE-SEP-A6KMR36ACIBYXLOJ -s 10.244.109.7/32 -m comment --comment \"newmyservice/monogo-mongodb:mongodb\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-A6KMR36ACIBYXLOJ -p tcp -m comment --comment \"newmyservice/monogo-mongodb:mongodb\" -m tcp -j DNAT --to-destination 10.244.109.7:27017\n-A KUBE-SEP-BG6TFBBQGAWE3Z7L -s 10.244.109.39/32 -m comment --comment \"kube-system/egressgateway-controller:webhook\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-BG6TFBBQGAWE3Z7L -p tcp -m comment --comment \"kube-system/egressgateway-controller:webhook\" -m tcp -j DNAT --to-destination 10.244.109.39:5822\n-A KUBE-SEP-EPZMDUCJDC2OHYAK -s 10.244.109.23/32 -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:grpc\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-EPZMDUCJDC2OHYAK -p tcp -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:grpc\" -m tcp -j DNAT --to-destination 10.244.109.23:3306\n-A KUBE-SEP-I4ZNX4OY6UO2PD2U -s 10.244.109.4/32 -m comment --comment \"kube-system/kube-dns:metrics\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-I4ZNX4OY6UO2PD2U -p tcp -m comment --comment \"kube-system/kube-dns:metrics\" -m tcp -j DNAT --to-destination 10.244.109.4:9153\n-A KUBE-SEP-IXUGJRA4GZQLZAJU -s 10.244.109.22/32 -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-IXUGJRA4GZQLZAJU -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http\" -m tcp -j DNAT --to-destination 10.244.109.22:80\n-A KUBE-SEP-JB5LSQZZDXLTL7QB -s 88.99.213.182/32 -m comment --comment \"calico-system/calico-typha:calico-typha\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-JB5LSQZZDXLTL7QB -p tcp -m comment --comment \"calico-system/calico-typha:calico-typha\" -m tcp -j DNAT --to-destination 88.99.213.182:5473\n-A KUBE-SEP-OEESFSBXALRV4VWG -s 10.244.109.8/32 -m comment --comment \"newmyservice/redis-master:tcp-redis\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-OEESFSBXALRV4VWG -p tcp -m comment --comment \"newmyservice/redis-master:tcp-redis\" -m tcp -j DNAT --to-destination 10.244.109.8:6379\n-A KUBE-SEP-RTIEMOUJKRROVH3J -s 10.244.109.22/32 -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-RTIEMOUJKRROVH3J -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls\" -m tcp -j DNAT --to-destination 10.244.109.22:443\n-A KUBE-SEP-SE3MTBCZ3RW6IAMO -s 10.244.109.4/32 -m comment --comment \"kube-system/kube-dns:dns\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-SE3MTBCZ3RW6IAMO -p udp -m comment --comment \"kube-system/kube-dns:dns\" -m udp -j DNAT --to-destination 10.244.109.4:53\n-A KUBE-SEP-TJ33L5TODYNVESW3 -s 88.99.213.182/32 -m comment --comment \"default/kubernetes:https\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-TJ33L5TODYNVESW3 -p tcp -m comment --comment \"default/kubernetes:https\" -m tcp -j DNAT --to-destination 88.99.213.182:6443\n-A KUBE-SEP-U2LOLC3TDNOYZEYN -s 10.244.109.5/32 -m comment --comment \"calico-apiserver/calico-api:apiserver\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-U2LOLC3TDNOYZEYN -p tcp -m comment --comment \"calico-apiserver/calico-api:apiserver\" -m tcp -j DNAT --to-destination 10.244.109.5:5443\n-A KUBE-SEP-UZI23UX3WZZ3NTLJ -s 10.244.109.20/32 -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-UZI23UX3WZZ3NTLJ -p udp -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp\" -m udp -j DNAT --to-destination 10.244.109.20:5353\n-A KUBE-SEP-VO2WZYMO5RK474PI -s 10.244.109.4/32 -m comment --comment \"kube-system/kube-dns:dns-tcp\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-VO2WZYMO5RK474PI -p tcp -m comment --comment \"kube-system/kube-dns:dns-tcp\" -m tcp -j DNAT --to-destination 10.244.109.4:53\n-A KUBE-SEP-WDTIMUUGW3CTM56Y -s 10.244.109.2/32 -m comment --comment \"kube-system/kube-dns:dns\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-WDTIMUUGW3CTM56Y -p udp -m comment --comment \"kube-system/kube-dns:dns\" -m udp -j DNAT --to-destination 10.244.109.2:53\n-A KUBE-SEP-X6NJHHYNJCAAN324 -s 10.244.109.23/32 -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:api\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-X6NJHHYNJCAAN324 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:api\" -m tcp -j DNAT --to-destination 10.244.109.23:8080\n-A KUBE-SEP-YFS4TYSYNIK72KIF -s 10.244.109.6/32 -m comment --comment \"calico-apiserver/calico-api:apiserver\" -j KUBE-MARK-MASQ\n-A KUBE-SEP-YFS4TYSYNIK72KIF -p tcp -m comment --comment \"calico-apiserver/calico-api:apiserver\" -m tcp -j DNAT --to-destination 10.244.109.6:5443\n-A KUBE-SERVICES -d 10.96.156.40/32 -p tcp -m comment --comment \"newmyservice/monogo-mongodb:mongodb cluster IP\" -j KUBE-SVC-UNMNSATZFYJDMRLS\n-A KUBE-SERVICES -d 10.96.10.82/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls cluster IP\" -j KUBE-SVC-7U7O5RDNYVDOUGBF\n-A KUBE-SERVICES -d 178.63.133.146/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls loadbalancer IP\" -j KUBE-EXT-7U7O5RDNYVDOUGBF\n-A KUBE-SERVICES -d 10.96.83.57/32 -p tcp -m comment --comment \"newmyservice/redis-master:tcp-redis cluster IP\" -j KUBE-SVC-MCYIF6SJCIAJCF25\n-A KUBE-SERVICES -d 10.96.194.177/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:grpc cluster IP\" -j KUBE-SVC-M3XNSA5ZTED24LO2\n-A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment \"default/kubernetes:https cluster IP\" -j KUBE-SVC-NPX46M4PTMTKRN6Y\n-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment \"kube-system/kube-dns:dns cluster IP\" -j KUBE-SVC-TCOU7JCQXEZGVUNU\n-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment \"kube-system/kube-dns:metrics cluster IP\" -j KUBE-SVC-JD5MR3NA4I4DYORP\n-A KUBE-SERVICES -d 10.96.155.64/32 -p tcp -m comment --comment \"calico-apiserver/calico-api:apiserver cluster IP\" -j KUBE-SVC-I24EZXP75AX5E7TU\n-A KUBE-SERVICES -d 10.96.194.177/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:api cluster IP\" -j KUBE-SVC-7CXIK4VIXYOGPYZ7\n-A KUBE-SERVICES -d 10.96.242.190/32 -p udp -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp cluster IP\" -j KUBE-SVC-NHXMRZ4FDHP77GE3\n-A KUBE-SERVICES -d 178.63.133.145/32 -p udp -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp loadbalancer IP\" -j KUBE-EXT-NHXMRZ4FDHP77GE3\n-A KUBE-SERVICES -d 10.96.10.82/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http cluster IP\" -j KUBE-SVC-2OFMA5BBWKIWDPSD\n-A KUBE-SERVICES -d 178.63.133.146/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http loadbalancer IP\" -j KUBE-EXT-2OFMA5BBWKIWDPSD\n-A KUBE-SERVICES -d 10.96.27.119/32 -p tcp -m comment --comment \"metallb-system/metallb-webhook-service cluster IP\" -j KUBE-SVC-GZ25SP4UFGF7SAVL\n-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment \"kube-system/kube-dns:dns-tcp cluster IP\" -j KUBE-SVC-ERIFXISQEP7F7OF4\n-A KUBE-SERVICES -d 10.96.34.83/32 -p tcp -m comment --comment \"calico-system/calico-typha:calico-typha cluster IP\" -j KUBE-SVC-RK657RLKDNVNU64O\n-A KUBE-SERVICES -d 10.96.226.99/32 -p tcp -m comment --comment \"kube-system/egressgateway-controller:webhook cluster IP\" -j KUBE-SVC-FTKCEC5VWOO5DYBC\n-A KUBE-SERVICES -m comment --comment \"kubernetes service nodeports; NOTE: this must be the last rule in this chain\" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS\n-A KUBE-SVC-2OFMA5BBWKIWDPSD ! -s 10.244.0.0/16 -d 10.96.10.82/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-2OFMA5BBWKIWDPSD -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http -> 10.244.109.22:80\" -j KUBE-SEP-IXUGJRA4GZQLZAJU\n-A KUBE-SVC-7CXIK4VIXYOGPYZ7 ! -s 10.244.0.0/16 -d 10.96.194.177/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:api cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-7CXIK4VIXYOGPYZ7 -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:api -> 10.244.109.23:8080\" -j KUBE-SEP-X6NJHHYNJCAAN324\n-A KUBE-SVC-7U7O5RDNYVDOUGBF ! -s 10.244.0.0/16 -d 10.96.10.82/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-7U7O5RDNYVDOUGBF -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls -> 10.244.109.22:443\" -j KUBE-SEP-RTIEMOUJKRROVH3J\n-A KUBE-SVC-ERIFXISQEP7F7OF4 ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p tcp -m comment --comment \"kube-system/kube-dns:dns-tcp cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment \"kube-system/kube-dns:dns-tcp -> 10.244.109.2:53\" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-66QAXE6EYTRCMRYX\n-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment \"kube-system/kube-dns:dns-tcp -> 10.244.109.4:53\" -j KUBE-SEP-VO2WZYMO5RK474PI\n-A KUBE-SVC-FTKCEC5VWOO5DYBC ! -s 10.244.0.0/16 -d 10.96.226.99/32 -p tcp -m comment --comment \"kube-system/egressgateway-controller:webhook cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-FTKCEC5VWOO5DYBC -m comment --comment \"kube-system/egressgateway-controller:webhook -> 10.244.109.39:5822\" -j KUBE-SEP-BG6TFBBQGAWE3Z7L\n-A KUBE-SVC-GZ25SP4UFGF7SAVL ! -s 10.244.0.0/16 -d 10.96.27.119/32 -p tcp -m comment --comment \"metallb-system/metallb-webhook-service cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-GZ25SP4UFGF7SAVL -m comment --comment \"metallb-system/metallb-webhook-service -> 10.244.109.13:9443\" -j KUBE-SEP-4HXEATJ23FYNZIVK\n-A KUBE-SVC-I24EZXP75AX5E7TU ! -s 10.244.0.0/16 -d 10.96.155.64/32 -p tcp -m comment --comment \"calico-apiserver/calico-api:apiserver cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-I24EZXP75AX5E7TU -m comment --comment \"calico-apiserver/calico-api:apiserver -> 10.244.109.5:5443\" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-U2LOLC3TDNOYZEYN\n-A KUBE-SVC-I24EZXP75AX5E7TU -m comment --comment \"calico-apiserver/calico-api:apiserver -> 10.244.109.6:5443\" -j KUBE-SEP-YFS4TYSYNIK72KIF\n-A KUBE-SVC-JD5MR3NA4I4DYORP ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p tcp -m comment --comment \"kube-system/kube-dns:metrics cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment \"kube-system/kube-dns:metrics -> 10.244.109.2:9153\" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-34ERLKPYC6DKDBGS\n-A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment \"kube-system/kube-dns:metrics -> 10.244.109.4:9153\" -j KUBE-SEP-I4ZNX4OY6UO2PD2U\n-A KUBE-SVC-M3XNSA5ZTED24LO2 ! -s 10.244.0.0/16 -d 10.96.194.177/32 -p tcp -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:grpc cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-M3XNSA5ZTED24LO2 -m comment --comment \"newmyservice/myservice-newshcan-shenvoy-svc:grpc -> 10.244.109.23:3306\" -j KUBE-SEP-EPZMDUCJDC2OHYAK\n-A KUBE-SVC-MCYIF6SJCIAJCF25 ! -s 10.244.0.0/16 -d 10.96.83.57/32 -p tcp -m comment --comment \"newmyservice/redis-master:tcp-redis cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-MCYIF6SJCIAJCF25 -m comment --comment \"newmyservice/redis-master:tcp-redis -> 10.244.109.8:6379\" -j KUBE-SEP-OEESFSBXALRV4VWG\n-A KUBE-SVC-NHXMRZ4FDHP77GE3 ! -s 10.244.0.0/16 -d 10.96.242.190/32 -p udp -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-NHXMRZ4FDHP77GE3 -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp -> 10.244.109.20:5353\" -j KUBE-SEP-UZI23UX3WZZ3NTLJ\n-A KUBE-SVC-NPX46M4PTMTKRN6Y ! -s 10.244.0.0/16 -d 10.96.0.1/32 -p tcp -m comment --comment \"default/kubernetes:https cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment \"default/kubernetes:https -> 88.99.213.182:6443\" -j KUBE-SEP-TJ33L5TODYNVESW3\n-A KUBE-SVC-RK657RLKDNVNU64O ! -s 10.244.0.0/16 -d 10.96.34.83/32 -p tcp -m comment --comment \"calico-system/calico-typha:calico-typha cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-RK657RLKDNVNU64O -m comment --comment \"calico-system/calico-typha:calico-typha -> 88.99.213.182:5473\" -j KUBE-SEP-JB5LSQZZDXLTL7QB\n-A KUBE-SVC-TCOU7JCQXEZGVUNU ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p udp -m comment --comment \"kube-system/kube-dns:dns cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment \"kube-system/kube-dns:dns -> 10.244.109.2:53\" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-WDTIMUUGW3CTM56Y\n-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment \"kube-system/kube-dns:dns -> 10.244.109.4:53\" -j KUBE-SEP-SE3MTBCZ3RW6IAMO\n-A KUBE-SVC-UNMNSATZFYJDMRLS ! -s 10.244.0.0/16 -d 10.96.156.40/32 -p tcp -m comment --comment \"newmyservice/monogo-mongodb:mongodb cluster IP\" -j KUBE-MARK-MASQ\n-A KUBE-SVC-UNMNSATZFYJDMRLS -m comment --comment \"newmyservice/monogo-mongodb:mongodb -> 10.244.109.7:27017\" -j KUBE-SEP-A6KMR36ACIBYXLOJ\n-A KUBE-SVL-2OFMA5BBWKIWDPSD -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:http -> 10.244.109.22:80\" -j KUBE-SEP-IXUGJRA4GZQLZAJU\n-A KUBE-SVL-7U7O5RDNYVDOUGBF -m comment --comment \"newmyservice/myservice-newshcan-envoy-env1:tls -> 10.244.109.22:443\" -j KUBE-SEP-RTIEMOUJKRROVH3J\n-A KUBE-SVL-NHXMRZ4FDHP77GE3 -m comment --comment \"newmyservice/myservice-newshcan-coredns-svc-lb:simple-udp -> 10.244.109.20:5353\" -j KUBE-SEP-UZI23UX3WZZ3NTLJ\n-A cali-OUTPUT -m comment --comment \"cali:GBTAv2p5CwevEyJm\" -j cali-fip-dnat\n-A cali-POSTROUTING -m comment --comment \"cali:Z-c7XtVd2Bq7s_hA\" -j cali-fip-snat\n-A cali-POSTROUTING -m comment --comment \"cali:nYKhEzDlr11Jccal\" -j cali-nat-outgoing\n-A cali-POSTROUTING -o vxlan.calico -m comment --comment \"cali:e9dnSgSVNmIcpVhP\" -m addrtype ! --src-type LOCAL --limit-iface-out -m addrtype --src-type LOCAL -j MASQUERADE --random-fully\n-A cali-PREROUTING -m comment --comment \"cali:r6XmIziWUJsdOK6Z\" -j cali-fip-dnat\n-A cali-nat-outgoing -m comment --comment \"cali:flqWnvo8yq4ULQLa\" -m set --match-set cali40masq-ipam-pools src -m set ! --match-set cali40all-ipam-pools dst -j MASQUERADE --random-fully\nCOMMIT\n# Completed on Wed Nov 6 15:16:23 2024\n","error":"current state of iptables","stacktrace":"github.com/spidernet-io/egressgateway/pkg/iptables.(Table).Apply\n\t/src/pkg/iptables/table.go:811\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).initApplyPolicy\n\t/src/pkg/agent/police.go:304\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).Reconcile.func1\n\t/src/pkg/agent/police.go:65\nsync.(Once).doSlow\n\t/usr/local/go/src/sync/once.go:74\nsync.(Once).Do\n\t/usr/local/go/src/sync/once.go:65\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).Reconcile\n\t/src/pkg/agent/police.go:62\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222"}

{"level":"error","ts":"2024-11-06T15:16:23.352Z","caller":"agent/police.go:67","msg":"init policy","error":"failed to apply rule nat: failed to program iptables, giving up after retries: exit status 2","stacktrace":"github.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).Reconcile.func1\n\t/src/pkg/agent/police.go:67\nsync.(Once).doSlow\n\t/usr/local/go/src/sync/once.go:74\nsync.(Once).Do\n\t/usr/local/go/src/sync/once.go:65\ngithub.com/spidernet-io/egressgateway/pkg/agent.(policeReconciler).Reconcile\n\t/src/pkg/agent/police.go:62\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/src/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222"}

lou-lan commented 5 days ago

helm get values $NAME --name $NAME will display the installation values. Are you looking to use IPv4 only?

amirdaaee commented 5 days ago

here it is:

USER-SUPPLIED VALUES:
feature:
  enableGatewayReplyRoute: true
  enableIPv6: true
  tunnelDetectMethod: interface=enp0s31f6

and no, im looking to use IPv6 only