search

loxilb-io / kube-loxilb

Implementation of kubernetes service load-balancer spec for loxilb
Apache License 2.0
85 stars 16 forks source link

The information display part of k8s service is wrong #48

Closed sadate closed 1 year ago

sadate commented 1 year ago

We want to request static ip LoadBalancer from ippool, according to the following configuration:

apiVersion: v1
kind: Service
metadata:
  name: sctp-svc
  annotations:
    loxilb.io/liveness: "yes"
    loxilb.io/ipam: "ipv4"
spec:
  loadBalancerClass: loxilb.io/loxilb
  externalTrafficPolicy: Local
  type: LoadBalancer
  externalIPs:
    - 123.123.123.88
    - 123.123.123.99
  selector:
    app.kubernetes.io/name: proxy
  ports:
  - name: name-of-service-port
    protocol: SCTP
    port: 5906
    targetPort: 5906

At this time, when we checked the service information, we found the following behavior:

$ kubectl get svc
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP                                                   PORT(S)           AGE
kubernetes   ClusterIP      10.96.0.1      <none>                                                        443/TCP           7d7h
sctp-svc     LoadBalancer   10.99.53.229   123.123.123.88,123.123.123.99,123.123.123.88,123.123.123.99   5906:32249/SCTP   22h

The display of the 'EXTERNAL-IP' part is repeated. Is this a bug? We are using the latest version of the docker image: ghcr.io/loxilb-io/kube-loxilb:latest

At this time, after testing, the working status of loxilb is normal, but the information display part of k8s service is wrong.

$ docker exec -it loxilb /bin/bash
# loxicmd get lb -o wide
|  EXTERNAL IP   | SECONDARY IPS | PORT | PROTOCOL | BLOCK | SELECT |  MODE   | ENDPOINT IP | TARGET PORT | WEIGHT |  STATE   |
|----------------|---------------|------|----------|-------|--------|---------|-------------|-------------|--------|----------|
| 123.123.123.88 |               | 5906 | sctp     |     0 | rr     | default | 10.0.0.201  |       32249 |     10 | active   |
| 123.123.123.99 |               | 5906 | sctp     |     0 | rr     | default | 10.0.0.201  |       32249 |     10 | active   |

# loxicmd get ep -o wide
|    HOST    |         NAME          | PTYPE | PORT  | DURATION | RETRIES | MINDELAY | AVGDELAY | MAXDELAY | STATE |
|------------|-----------------------|-------|-------|----------|---------|----------|----------|----------|-------|
| 10.0.0.201 | 10.0.0.201_sctp_32249 | sctp: | 32249 |       60 |       2 |          |          |          | ok    |
TrekkieCoder commented 1 year ago

Thanks a lot for the raising the issue. Yes there was a display problem due to how kube-loxilb maintained static externalIP. But there have been many fixes in kube-loxilb related to IPAM. If you update to the latest, you should not see this issue. (It is verified in the latest image as of writing.)

apiVersion: v1
kind: Service
metadata:
  name: sctp-lb-fullnat
  annotations:
   loxilb.io/liveness: "yes"
   loxilb.io/lbmode: "fullnat"
spec:
  loadBalancerClass: loxilb.io/loxilb
  externalTrafficPolicy: Local
  selector:
    what: sctp-fullnat-test
  ports:
    - port: 57004
      protocol: SCTP
      targetPort: 9999
  type: LoadBalancer
  externalIPs:
    - 123.123.123.88
    - 123.123.123.99

The corresponding display :

sudo kubectl get svc
NAME              TYPE           CLUSTER-IP      EXTERNAL-IP                     PORT(S)            AGE
kubernetes        ClusterIP      172.17.0.1      <none>                          443/TCP            31m
sctp-lb-default   LoadBalancer   172.17.3.196    llb-123.123.123.1               55004:30944/SCTP   17m
sctp-lb-fullnat   LoadBalancer   172.17.63.166   123.123.123.88,123.123.123.99   57004:31532/SCTP   2s
sctp-lb-onearm    LoadBalancer   172.17.63.207   llb-123.123.123.1               56004:30163/SCTP   17m
tcp-lb-default    LoadBalancer   172.17.38.207   llb-123.123.123.1               55002:31902/TCP    17m
tcp-lb-fullnat    LoadBalancer   172.17.53.90    llb-123.123.123.1               57002:31264/TCP    17m
tcp-lb-onearm     LoadBalancer   172.17.44.212   llb-123.123.123.1               56002:31291/TCP    17m
udp-lb-default    LoadBalancer   172.17.19.86    llb-123.123.123.1               55003:30120/UDP    17m
udp-lb-fullnat    LoadBalancer   172.17.61.76    llb-123.123.123.1               57003:32140/UDP    17m
udp-lb-onearm     LoadBalancer   172.17.48.87    llb-123.123.123.1               56003:30222/UDP    17m

We would cut a stable release version once we are able to fix all currently reported outstanding (major) issues.

sadate commented 1 year ago

Thank you very much for the timely response. After our testing, after re-pulling the latest image, the fault has disappeared.

$ kubectl get svc
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP                     PORT(S)           AGE
kubernetes   ClusterIP      10.96.0.1      <none>                          443/TCP           7d8h
sctp-svc     LoadBalancer   10.108.36.82   123.123.123.88,123.123.123.99   5906:30981/SCTP   3s