Inconsistent route choice between gateway and sidecar under HTTPRoutes with same parent

qts0312 commented 7 months ago

What is the issue?

Here is the configuration, traffic from sidecar chooses hr-0 and goes to the ratings service, while traffic from gateway chooses hr-1 and goes to the productpage service, implying inconsistency between the route choice of sidecar and gateway.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: hr-0
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: gw
  - group: core
    port: 80
    name: back
    kind: Service
  hostnames:
  - www.bookinfo.com
  rules:
  - backendRefs:
    - name: ratings
      port: 9090
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: hr-1
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: gw
  - group: core
    port: 80
    name: back
    kind: Service
  hostnames:
  - www.bookinfo.com
  rules:
  - backendRefs:
    - name: productpage
      port: 9080

How can it be reproduced?

Create services ratings, productpage and back in the proper ports, and then create gateway as below.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: gw
spec:
  gatewayClassName: contour
  listeners:
  - name: port-0
    port: 9080
    protocol: HTTP
    hostname: www.bookinfo.com
    allowedRoutes:
      namespaces:
        from: All

curl -H"host:www.bookinfo.com" http://back.default.svc.cluster.local:80 from one of the pods will go to ratings service, and curl -H"host:www.bookinfo.com" http://<gateway service external ip>:9080 from gateway will go to productpage service.

Logs, error output, etc

Behavior is described above, and details can be provided if need.

output of `linkerd check -o short`

Status check results are √

Environment

Kubernetes Version: 1.28.2
Linkerd Version: stable-2.14.10
Gateway Implementation: Contour

Possible solution

No response

Additional context

No response

Would you like to work on fixing this bug?

None

kflynn commented 6 months ago

Which Route has the oldest creation date? (This is a Gateway API route-merging question, and the creation date matters for that.)

qts0312 commented 6 months ago

I think vs-0 is created first, but what confuses me is that traffic from sidecar and gateway chooses different HTTPRoute.

kflynn commented 6 months ago

Did you mean hr-0?

The thing to realize here is that Contour and Linkerd are independently processing all the HTTPRoutes they find, and their implementations share absolutely nothing beyond the Gateway API spec. They're both supposed to treat HTTPRoutes exactly the same way, but there's potential for issues in either.

In this case, you have two single-rule HTTPRoutes that are identical except for the backendRef. The Gateway API spec dictates that a conformant implementation, in that case, has to obey the HTTPRoute that was created first – so if hr-0 is created first, it would seem that Linkerd is doing the right thing.

One other question: what version of Gateway API do you have installed?

kflynn commented 4 months ago

Hey @qts0312, any more info on your Gateway API version?

qts0312 commented 4 months ago

Sorry for the late reply. My Gateway API version is experimental channel v1.0.0.

sunjayBhatia commented 2 months ago

What version of Contour are you using @qts0312 ?

In Contour 1.29.0+ we merged/released some changes to fix the conflict handling when there are identical matches between routes: https://github.com/projectcontour/contour/pull/6188

linkerd / linkerd2