search

udevbe / greenfield

HTML5 Wayland compositor :seedling:
GNU Affero General Public License v3.0
901 stars 27 forks source link

Use WebTransport instead of WebSocket #107

Open Zubnix opened 1 year ago

Zubnix commented 1 year ago

Compositor-proxy currently uses WebSockets to communicate application [video] frame data. This has well known drawbacks for use in real-time communication as it's essentially a TCP connection. Instead we should use WebTransport which offers UDP semantics which is far better suited for real-time communication.

Initial preparing work has already been completed by switching to the uwebsocketjs library which will eventually support WebTransport

ehfd commented 1 year ago

Thanks for the information! Looking forward for its maturation.

Zubnix commented 1 year ago

fyi webrtc data-channels have been implemented and subsequently dropped for being too slow because of the SCTP congestion control overhead. That is, on a 300Mbit connection with zero packet loss, you'd get 30Mbit throughput which is unacceptably slow.

So we patiently await the support and arrival of WebTransport.

ehfd commented 11 months ago

https://caniuse.com/webtransport

https://caniuse.com/webcodecs

All major browsers other than Safari now support WebTransport. Firefox is working on WebCodecs.

ehfd commented 8 months ago

https://github.com/selkies-project/selkies-gstreamer/issues/48#issuecomment-1764124875

We've been investigating WebTransport but there are a few catches outlined above (requiring valid HTTPS certificates and other issues).

But overall, the web browser stack is ready.

More references: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1571 https://github.com/kixelated/moq-rs https://github.com/wanjohiryan/warp https://github.com/wanjohiryan/Arc3dia

ehfd commented 8 months ago

fyi webrtc data-channels have been implemented and subsequently dropped for being too slow because of the SCTP congestion control overhead. That is, on a 300Mbit connection with zero packet loss, you'd get 30Mbit throughput which is unacceptably slow.

We (https://github.com/selkies-project/selkies-gstreamer) use WebRTC DataChannels for passing user input and clipboard, and use the native WebRTC MediaStreams for video and video. It's relatively inflexible but more usable than using DataChannels for everything.