Websocket support

[johnharris85]

Description

Currently our documented solution for websockets support is to designate the service as tcp: https://kuma.io/docs/1.6.x/policies/protocol-support-in-kuma/#websocket-support. However this discounts services that may provide websocket and regular services (requiring an http designation for various policies / functionality to work).

Is there a way we could provide websocket support with an http service leveraging Envoy's upgrade_config?

How can we support this on MeshGateway?

Sample ProxyTemplate to achieve this below:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: ws-upgrade-out
spec:
  selectors:
    - match:
        kuma.io/service: '*'
  conf:
    imports:
      - default-proxy
    modifications:
      - networkFilter:
          operation: patch
          match:
            name: envoy.filters.network.http_connection_manager
          value: |
            name: envoy.filters.network.http_connection_manager
            typed_config:
              '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              upgrade_configs:
                - upgrade_type: websocket
              http2_protocol_options:
                allow_connect: true

This is slightly simplified (the http2_protocol_options really only need to be on the inbound HCMs). Also there are two options w.r.t. protocol options. Because HTTP2 doesn't support websocket upgrade, you either need to set the outbound cluster to http_protocol_options (to force HTTP1.1) or set allow_connect on the http2_protocol_options to allow HTTP2 between Envoy's but allow it to use CONNECT protocol to essentially tunnel it through to the eventual upstream. Depends on the use case / requirements as to which you'd use I guess :shrug:

[lahabana]

Triage: @bartsmykla looked at this in the past and he believes he didn't go this way because he wasn't aware of allow_connect. This suggestion looks great just need to make sure all cases still work. Also this would be the default for all inbounds with kuma.io/protocol: http. In any case we need good testing of this.

[github-actions[bot]]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[chris-aeviator]

I'm currently troubleshooting a graphql-ws connection (and thus ended up here), I have set it up as a TCP service but I can sucessfully use the connection for HTTP as well for normal graphql HTTP calls (note the node.js server splits it to two different clients).

Is this issue meant to push for full support of http rather than limited support, or do you don't see yourself able to use http & websocket at the same time at all?

[lahabana]

That's a very good point @chris-aeviator maybe ability to upgrade to ws is more of a route level parameter on an http service. @johnharris85 what's your opinion?

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[slonka]

re-pinging @johnharris85: what's your opinion?

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions[bot]]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[slonka]

triage: xrel https://gateway-api.sigs.k8s.io/geps/gep-1911/#api-semantics