Support DirectResponse with K8S Gateway API

[DuncanDoyle]

Gloo Edge Product

Enterprise

Gloo Edge Version

1.17.0

Is your feature request related to a problem? Please describe.

The Gloo Edge API has a feature called Direct Response Action that allows you to send a direct response to a client on a route match, without hitting the upstream.

In our K8S Gateway API we currently don't support this feature. There is an ongoing discussion in community to add this functionality as a first class citizen in K8S Gateway API in the form of an HttpRouteFilter: https://github.com/kubernetes-sigs/gateway-api/issues/2826

Describe the solution you'd like

Suppport DirectResponseAction semantics with K8S Gateway API, ideally based on features in the Gateway API spec itself: https://github.com/kubernetes-sigs/gateway-api/issues/2826

Describe alternatives you've considered

Implement DirectResponseAction in a Solo API as an extension to K8S Gateway API.

Additional Context

No response

┆Issue is synchronized with this Asana task by Unito

[timflannagan]

I'm going to take this over from Tyler. I created an internal onepager that outlines a couple of different solutions: https://soloio.slab.com/posts/support-direct-response-with-k-8-s-gateway-api-cdlysioo#hzi0v-problem-statement. I'm still onboarding to the repository, but I was hacking on a POC that extends the RouteOptions API and that seems to be working pretty smoothly when I test it out locally:

```yaml --- kind: Gateway apiVersion: gateway.networking.k8s.io/v1 metadata: name: http spec: gatewayClassName: gloo-gateway listeners: - protocol: HTTP port: 8080 name: http allowedRoutes: namespaces: from: All --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httpbin namespace: httpbin labels: example: httpbin-route spec: hostnames: - "www.example.com" parentRefs: - name: http namespace: gloo-system rules: - backendRefs: - name: httpbin port: 8000 matches: - headers: - name: Authorization type: RegularExpression value: "Bearer .*" - backendRefs: - name: httpbin port: 8000 filters: - type: ExtensionRef extensionRef: group: gateway.solo.io kind: RouteOption name: gloo-direct-option --- apiVersion: gateway.solo.io/v1 kind: RouteOption metadata: name: gloo-direct-option namespace: httpbin spec: options: directResponse: status: 401 body: "Unauthorized: No token provided" ```

$ curl -i localhost:8080/headers -H "host: www.example.com"                                
Handling connection for 8080
HTTP/1.1 401 Unauthorized
content-length: 39
content-type: text/plain
date: Wed, 14 Aug 2024 21:44:01 GMT
server: envoy

value:"Unauthorized: No token provided"%                                                                                                       
$ curl -i localhost:8080/headers -H "host: www.example.com" -H "Authorization: Bearer asdf"
Handling connection for 8080
HTTP/1.1 200 OK
access-control-allow-credentials: true
access-control-allow-origin: *
content-type: application/json; encoding=utf-8
date: Wed, 14 Aug 2024 21:43:57 GMT
content-length: 381
x-envoy-upstream-service-time: 0
server: envoy
...

[EItanya]

I was debating commenting in Slab, but I think this is the best spot. It's outlined there that adding this to the RouteOptions may create a confusing mix of responsibilities for that API. In the classic API what would this field do? I don't want to require a separate CRD for this, but I actually think given the constraints it may be the best path forward. Realistically this is something we could get the upstream API to accept, and most likely would come in the form of a new Filter, given that I don't think it's worth muddying up RouteOptions with more non-policy APIs.

We could even have this new CRD be Alpha with zero plan to graduate it given that we're pushing upstream to take it. What do others think?

[timflannagan]

@EItanya Introducing an alpha CRD also SGTM. We could either introduce a generic "RouteActions" API or a dedicated API for this functionality. I think I'm leaning towards the latter since (afaik) most of the route action functionality is already present in the upstream HTTPRoute API. WYDT? cc @lgadban

[npolshakova]

I think we either need a temporary API that will be replaced with upstream's direct response, or we should just design "RouteActions" fully and account for other use cases (single destinations that can specify the destinationSpec, multi destinations, and upstreamGroups).

[timflannagan]

I was talking with eitan and lawrence offline and I think we're leaning towards the temporary API still, but lawrence made some good points about the Upstream approach, and I think we still need to reach consensus on which design option we'll end up pursuing.

or we should just design "RouteActions" fully and account for other use cases

Good point. I have less context on the use cases you mentioned and I haven't went through your slab design doc.

[timflannagan]

Met with eitan/nina/lawrence/sam on Monday and determined that we'd like to introduce a new API for configuring direct response actions. For more information on this decision, see the finalized solution section for the slab doc.

I'm working on implementing this as a route plugin this week. Here's the API surface I had in mind:

// DirectResponseRouteSpec describes the desired state of a DirectResponseRoute.
//
// +kubebuilder:validation:XValidation:message="The 'body' field is required when 'code' is a 2xx status code",rule="self.code < 200 || self.code >= 300 || has(self.body)"
type DirectResponseRouteSpec struct {
    // Status defines the HTTP status code to return for this route.
    //
    // +kubebuilder:validation:Required
    // +kubebuilder:validation:Minimum=100
    // +kubebuilder:validation:Maximum=599
    Status *uint32 `json:"code"`
    // Body defines the content to be returned in the HTTP response body.
    // The maximum length of the body is restricted to prevent excessively large responses.
    //
    // +kubebuilder:validation:MaxLength=4096
    // +kubebuilder:validation:Optional
    Body *string `json:"body,omitempty"`
}

There's some open questions that I still need to burn down w.r.t the right UX/behavior for this resource:

• Do we need a targetRef field in the API surface, e.g. RouteOptions, to support bidirectional binding?
• Do we support cross-namespace references via ReferenceGrants, or we only support an HTTPRoute referencing a DRR resource that lives in the same namespace?
• Do we need to populate the DRR resources' status, or would populating the HTTPRoute's status with a condition that signals to the user that a DRR has been successfully applied to the output route

cc @DuncanDoyle

[timflannagan]

Need to add the category:gloo-gateway kubebuilder marker to make it easy to discover this resource too. See https://github.com/solo-io/solo-projects/issues/6605 for more information on this.