knative / serving

Kubernetes-based, scale-to-zero, request-driven compute
https://knative.dev/docs/serving/
Apache License 2.0
5.54k stars 1.15k forks source link

Figure out a plan for https://github.com/gorilla/websocket #13824

Closed dprotaso closed 1 year ago

dprotaso commented 1 year ago

https://github.com/gorilla/websocket is deprecated and unmaintained.

We should switch to something that is and ensure everything works as expected (ie. draining in queue proxy etc).

dprotaso commented 1 year ago

/triage accepted

dprotaso commented 1 year ago

Possible alternative - https://github.com/gobwas/ws

Gekko0114 commented 1 year ago

Hi @dprotaso , Can I work on this if you don't plan to do it?

dprotaso commented 1 year ago

@Gekko0114 sure

Gekko0114 commented 1 year ago

Thanks! /assign

Gekko0114 commented 1 year ago

We have to fix knative/pkg as well. https://github.com/knative/pkg/issues/2727

skonto commented 1 year ago

@Gekko0114 @dprotaso pls check: https://github.com/knative/serving/pull/13932#issuecomment-1542158365

dprotaso commented 1 year ago

From: @AlexVulaj

Hey all - sorry to jump in here as I've been informed of this work by @skonto . I'm not sure if this changes your plans for this work at all, but a group I've started at Red Hat is currently in active discussions with the original maintainers of the Gorilla Web Toolkit to adopt the project and reboot it. While there are no hard timelines at this time, feel free to contact me if you have any questions or would like to know more.

I'm not really married to any implementation - in theory I'd prefer the go stdlib just support and maintain websocket as a first party https://pkg.go.dev/golang.org/x/net/websocket

Oddly what's funny that page links out to https://pkg.go.dev/nhooyr.io/websocket

Also sorta random is the lack of WebSocket support over HTTP/2 https://github.com/golang/go/issues/32763#issuecomment-931127991 & https://github.com/golang/go/issues/49918

skonto commented 1 year ago

I'm not really married to any implementation - in theory I'd prefer the go stdlib just support

In the past they referenced gorilla in their docs. Now just check updates for https://github.com/nhooyr/websocket does not seem that active either. I am not in favor of anything, however before we do any change pls consider the info for the reboot.

dprotaso commented 1 year ago

Yeah nhooyr doesn't seem active - which is unfortunate given security focused PRs https://github.com/nhooyr/websocket/pull/360

dprotaso commented 1 year ago

The other bit https://github.com/gobwas/ws mentions in their README.md

This implementation of RFC6455 passes Autobahn Test Suite and currently has about 78% coverage.

I wonder what kind of coverage gorilla/websocket has

dprotaso commented 1 year ago

When things are decided between RedHat and the gorilla folks do you mind following up on this issue

/assign @skonto @AlexVulaj

knative-prow[bot] commented 1 year ago

@dprotaso: GitHub didn't allow me to assign the following users: AlexVulaj.

Note that only knative members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see the contributor guide

In response to [this](https://github.com/knative/serving/issues/13824#issuecomment-1548041321): >When things are decided between RedHat and the gorilla folks do you mind following up on this issue > >/assign @skonto @AlexVulaj Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
dprotaso commented 1 year ago

@Gekko0114 thanks for the help in the meantime

cristaloleg commented 1 year ago

The other bit gobwas/ws mentions in their README.md

This implementation of RFC6455 passes Autobahn Test Suite and currently has about 78% coverage.

I wonder what kind of coverage gorilla/websocket has

Hey, gobwa/ws maitainer here. I have plans to looks at autobahn tests in June, so coverage should increase. No official deadline for now.

andig commented 1 year ago

Worth mentioning https://github.com/nhooyr/websocket too

ReToCode commented 1 year ago

@dprotaso might be off-topic, but why do we use Websockets between Activator <> Autoscaler in the first place? Could that not just be GRPC?

skonto commented 1 year ago

The project is back alive: https://github.com/gorilla/websocket/commit/931041c5ee6de24fe9cba1aa16f1a0b910284d6d.

skonto commented 1 year ago

@ReToCode We changed the stats format for QP <-> Autoscaler long ago to be pb: https://github.com/knative/serving/pull/8396, https://github.com/knative/serving/pull/8556. Making more efficient than using json etc. We use websockets with a binary format for the Activator <-> Autoscaler exchange and use http based scraping for Autoscaler <-> QP (qp sets "application/protobuf" in the response to indicate that it sends pb data). Probably we could have grpc/http2 there assuming it performs better but not sure about it for this use case. @dprotaso could add more.

dprotaso commented 1 year ago

Probably we could have grpc/http2 there assuming it performs better but not sure about it for this use case.

Yeah prior to making these changes we'd want to use the new perf tests to see what gains we get. Plus we now have to play a 'migration' path if we do make changes

From the historical point of view I don't recall why we went with websockets+json - maybe it was easier to start with.

dprotaso commented 1 year ago

Since we're already using the latest version of https://github.com/gorilla/websocket I'm going to close this out

I think churning on websocket libraries won't be particularly useful - but rather work towards doing something more efficient. But we should open a separate issue for that.