get the opposite result

[SolarisYan]

Hi @jackkleeman, After I deploymented this operator, I get the opposite result.I can't access baidu.com, but others site can. I expect that i can only access baidu.com. How can i debug this, thanks.

kubectl -n egress-operator-system get po NAME READY STATUS RESTARTS AGE baidu-6499d5495b-g5t87 1/1 Running 0 54m egress-operator-controller-manager-6f99db96f4-v45k5 2/2 Running 0 8d

kubectl -n egress-operator-system logs baidu-6499d5495b-g5t87 ...

[2020-02-01T08:48:59.603Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-" [2020-02-01T08:49:09.602Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-" [2020-02-01T08:49:19.602Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-" [2020-02-01T08:49:29.602Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-" ...

kubectl -n egress-operator-system logs egress-operator-controller-manager-6f99db96f4-v45k5 -c manager ... 2020-02-01T07:39:38.361Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"} 2020-02-01T07:39:38.460Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"} 2020-02-01T07:39:53.271Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"} 2020-02-01T07:40:59.660Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"} ...

kubectl -n egress-operator-system get networkpolicy NAME POD-SELECTOR AGE baidu egress.monzo.com/gateway=baidu 67m egress-operator-public-egress app=egress-gateway 8d

the CoreDNS config: .:53 { errors health egressoperator egress-operator-system cluster.local kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa } prometheus :9153 forward . /etc/resolv.conf cache 30 reload loadbalance }

the ExternalService yaml:

apiVersion: egress.monzo.com/v1
kind: ExternalService
metadata:
  name: baidu
spec:
  dnsName: baidu.com
  hijackDns: true
  ports:
  - port: 80
  - port: 443
    protocol: TCP
  minReplicas: 1
  maxReplicas: 3
  targetCPUUtilizationPercentage: 30
  resources:
    requests:
      cpu: 100m
      memory: 100Mi
    limits:
      cpu: 500m
      memory: 200Mi

the pod yaml:

kind: Pod
apiVersion: v1
metadata:
  name: test-pod
  labels:
    egress.monzo.com/gateway: baidu
spec:
  containers:
  - name: test-pod
    image: alpine
    command:
    - "/bin/sleep"
    args:
    - "36000"
  restartPolicy: "Never"

[jackkleeman]

This operator won't block other sites for you, you'll need a default deny egress network policy which allows internal traffic but not external traffic, in namespaces you want to control. If you do that, then only traffic through gateways (which looks like an internal IP) will be allowed

I've updated the readme to give an example default deny egress policy

[jackkleeman]

I see the problem - you have the wrong label on your pod - should be egress.monzo.com/allowed-gateway: baidu

[SolarisYan]

Hi @jackkleeman I have followed as you said, but now i also can't access the external server like baidu.com

[SolarisYan]

I find that in baidu-6499d5495b-g5t87, i can't access the dns, and i can't access any site, include the host ip.

[SolarisYan]

I have use cilium to solve it

[jackkleeman]

I suspect that your network policy was too strict and it wasn't allowing internal traffic.