field ingress.spec.defaultBackend.service.port.number is broken

ahus1 commented 4 days ago

What happened:

I've set up Minikube with nginx ingress and ssl-passthrough. When I specify in the ingress the port name it works, but it doesn't work when specifying the port number.

I0628 12:18:46.971157       7 nginx.go:804] "Handling TCP connection" remote="192.168.39.1:42486" local="10.244.0.14:443"
I0628 12:18:46.973113       7 tcp.go:74] "TLS Client Hello" host="keycloak.keycloak-namespace.192.168.39.71.nip.io"
I0628 12:18:46.973132       7 tcp.go:84] "passing to" hostport="10.104.89.104:0"
E0628 12:18:46.995568       7 tcp.go:87] "error dialing proxy" err="dial tcp 10.104.89.104:0: connect: connection refused" ip="10.104.89.104" port=0 hostname="keycloak.keycloak-namespace.192.168.39.71.nip.io"

What you expected to happen:

I expected specifying a port number in the Ingress would work as well.

$ kubectl explain ingress.spec.defaultBackend.service.port
KIND:     Ingress
VERSION:  networking.k8s.io/v1

RESOURCE: port <Object>

DESCRIPTION:
     port of the referenced service. A port name or port number is required for
     a IngressServiceBackend.

     ServiceBackendPort is the service port being referenced.

FIELDS:
   name <string>
     name is the name of the port on the Service. This is a mutually exclusive
     setting with "Number".

   number       <integer>
     number is the numerical port number (e.g. 80) on the Service. This is a
     mutually exclusive setting with "Name".

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

NGINX Ingress controller
  Release:       v1.10.1
  Build:         4fb5aac1dd3669daa3a14d9de3e3cdb371b4c518
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.25.3

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.2", GitCommit:"5835544ca568b757a8ecae5c153f317e5736700e", GitTreeState:"clean", BuildDate:"2022-09-21T14:33:49Z", GoVersion:"go1.19.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"30", GitVersion:"v1.30.0", GitCommit:"7c48c2bd72b9bf5c44d21d7338cc7bea77d0ad2a", GitTreeState:"clean", BuildDate:"2024-04-17T17:27:03Z", GoVersion:"go1.22.2", Compiler:"gc", Platform:"linux/amd64"}

Environment: minikube version: v1.33.1

How to reproduce this issue:

Ingress that doesn't work (note that "port.number" is set)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    javaoperatorsdk.io/previous: 4a06bec1-adbc-4a56-b22d-13540a64baff
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
  creationTimestamp: "2024-06-28T10:38:07Z"
  generation: 8
  labels:
    app: keycloak
    app.kubernetes.io/instance: keycloak-kubernetes-quickstart
    app.kubernetes.io/managed-by: keycloak-operator
  name: keycloak-kubernetes-quickstart-ingress
  namespace: keycloak-namespace
  ownerReferences:
  - apiVersion: k8s.keycloak.org/v2alpha1
    kind: Keycloak
    name: keycloak-kubernetes-quickstart
    uid: 7adb441e-f3b2-46a4-9429-e67bf7ffc534
  resourceVersion: "6716"
  uid: a0972d87-f144-4151-818a-bfff7ead1b94
spec:
  defaultBackend:
    service:
      name: keycloak-kubernetes-quickstart-service
      port:
        number: 8443
  ingressClassName: nginx
  rules:
  - host: keycloak.keycloak-namespace.192.168.39.71.nip.io
    http:
      paths:
      - backend:
          service:
            name: keycloak-kubernetes-quickstart-service
            port:
              number: 8443
        path: /
        pathType: Prefix
status:
  loadBalancer:
    ingress:
    - ip: 192.168.39.71

Ingress that works (note that "port.name" is set):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    javaoperatorsdk.io/previous: 4a06bec1-adbc-4a56-b22d-13540a64baff
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
  creationTimestamp: "2024-06-28T10:38:07Z"
  generation: 9
  labels:
    app: keycloak
    app.kubernetes.io/instance: keycloak-kubernetes-quickstart
    app.kubernetes.io/managed-by: keycloak-operator
  name: keycloak-kubernetes-quickstart-ingress
  namespace: keycloak-namespace
  ownerReferences:
  - apiVersion: k8s.keycloak.org/v2alpha1
    kind: Keycloak
    name: keycloak-kubernetes-quickstart
    uid: 7adb441e-f3b2-46a4-9429-e67bf7ffc534
  resourceVersion: "7351"
  uid: a0972d87-f144-4151-818a-bfff7ead1b94
spec:
  defaultBackend:
    service:
      name: keycloak-kubernetes-quickstart-service
      port:
        name: https
  ingressClassName: nginx
  rules:
  - host: keycloak.keycloak-namespace.192.168.39.71.nip.io
    http:
      paths:
      - backend:
          service:
            name: keycloak-kubernetes-quickstart-service
            port:
              number: 8443
        path: /
        pathType: Prefix
status:
  loadBalancer:
    ingress:
    - ip: 192.168.39.71

Service

apiVersion: v1
kind: Service
metadata:
  annotations:
    javaoperatorsdk.io/previous: 2622db6d-9abc-4b1b-94fc-f04b6c27a41c
  creationTimestamp: "2024-06-28T10:38:07Z"
  labels:
    app: keycloak
    app.kubernetes.io/instance: keycloak-kubernetes-quickstart
    app.kubernetes.io/managed-by: keycloak-operator
  name: keycloak-kubernetes-quickstart-service
  namespace: keycloak-namespace
  ownerReferences:
  - apiVersion: k8s.keycloak.org/v2alpha1
    kind: Keycloak
    name: keycloak-kubernetes-quickstart
    uid: 7adb441e-f3b2-46a4-9429-e67bf7ffc534
  resourceVersion: "708"
  uid: bc3408f7-da8f-4441-bc2d-5949698c69b1
spec:
  clusterIP: 10.104.89.104
  clusterIPs:
  - 10.104.89.104
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: https
    port: 8443
    protocol: TCP
    targetPort: 8443
  - name: management
    port: 9000
    protocol: TCP
    targetPort: 9000
  selector:
    app: keycloak
    app.kubernetes.io/instance: keycloak-kubernetes-quickstart
    app.kubernetes.io/managed-by: keycloak-operator
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Configuration logged by Nginx. You'll see that in one of the configs the PassthroughBackends has set port 0 when it is broken, and a port 8443 when it works.

longwuyuan commented 4 days ago

/triage accepted This is a similar to of 9030

Unfortunately, what is know is that the field "ingress.spec.defaultBackend" is not working. Since this issue is about a spec further down in that field, we can conclude that the triage for this issue is accepted.

Because the flag "--default-backend-service" passed to the controller works now, it can be considered a workaround.

There is some info in that other issue that the problem was caused by this PR https://github.com/kubernetes/ingress-nginx/pull/8825 . I will create a issue to explore reverting what that PR changed

@Gacko any comments

@rikatz @strongjz @tao12345666333 if it is as simple as reverting https://github.com/kubernetes/ingress-nginx/pull/8825, would you want to review/consider it

longwuyuan commented 4 days ago

@harry1064 any comments

longwuyuan commented 4 days ago

/retitle field ingress.spec.defaultBackend.service.port.number is broken

kubernetes / ingress-nginx

field ingress.spec.defaultBackend.service.port.number is broken #11517