nirmata / kube-static-egress-ip

Kubernetes CRD to manage static egress IP addresses for workloads
Apache License 2.0
213 stars 38 forks source link

Failed to setup routes #28

Open lelvisl opened 5 years ago

lelvisl commented 5 years ago

logs

I0723 19:18:46.507067       1 director.go:102] Created ipset name: EGRESS-IP-XFVL3XZHQWBKAPWE
I0723 19:18:46.507676       1 director.go:111] Added ips [192.168.7.10] to the ipset name: EGRESS-IP-XFVL3XZHQWBKAPWE
I0723 19:18:46.508735       1 director.go:127] iptables rule in mangle table PREROUTING chain to match src to ipset
E0723 19:18:46.522160       1 controller.go:286] Failed to setup routes to send the egress traffic to gateway due to Failed to add route in custom route table due to: exit status 2
I0723 19:18:46.522179       1 controller.go:199] Successfully synced 'resequip/test'

manifest

apiVersion: staticegressips.nirmata.io/v1alpha1
kind: StaticEgressIP
metadata:
  name: test
spec:
  rules:
  - egressip: 10.35.12.65
    service-name: backend
    cidr: 10.35.12.17/32

service

apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"backend"},"name":"backend","namespace":"resequip"},"spec":{"ports":[{"name":"http","port":80}],"selector":{"app":"backend"}}}
    metallb.universe.tf/address-pool: oebs
  creationTimestamp: "2019-07-12T01:43:53Z"
  labels:
    app: backend
  name: backend
  namespace: resequip
  resourceVersion: "249156023"
  selfLink: /api/v1/namespaces/resequip/services/backend
  uid: 81891851-a446-11e9-96c8-0050562c0156
spec:
  clusterIP: 192.168.241.54
  externalTrafficPolicy: Local
  healthCheckNodePort: 32592
  loadBalancerIP: 10.35.12.65
  ports:
  - name: http
    nodePort: 30342
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: backend
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 10.35.12.65
tyu38 commented 4 years ago

I have the same problem, someone help?

mangermaier commented 4 years ago

same problem here, any news about this?

Jancis commented 4 years ago

I think i debugged it to this line: https://github.com/nirmata/kube-static-egress-ip/blob/master/pkg/director/director.go#L148 Connected to container, tried to add routes manually, but ip route yields RTNETLINK answers: Network is unreachable. even though i am able to ping the host (same network) and even connect via ssh. I start to think it's some GKE specific thing (i am using GKE for this), probably the node subnet address does not allow setting the other node as gateway for the route.