yume-chan
commented
4 months ago Latency is coming from two locations:
- Network latency between sharing host and client
- Lag behind in video rendering
The first one is out of our control. But because the transfer protocol is using Comlink package, each read call needs to send one request packet from client to host, then one response packet back. This is for properly maintaining back pressure in streams. Now I changed it to a simpler protocol, the host sends packets continuously unless the client sends a flow control command. It doesn't have perfect back pressure but doesn't wait for one packet to arrive on client before sending the next one, so it's much better for high latency environments. I also added a network stats panel to show raw network latency.
The second one is caused by slow video rendering. On my PC with a discrete GPU, the canvas 2d renderer can decode and render ~1200 720P video frames per second. But a high end Snapdragon 8 Gen 1+ can only produce ~110 frames per second (even lower for higher resolutions). Switching to image bitmap renderer increases that to ~130, and switching to WebGL increases that to ~270, which should be enough for rendering native 1440P resolution screen from many high end phones.
In summary, check the network latency in network stats panel first. If that is low but the screen mirror video is still laggy, try lowering the resolution and/or frame rate limiter.
Pre-issue checklist
The environment Tango is running in
Microsoft Edge 123
The device you're connecting to
Not related
What did you do and what happened?
Got a report from Discord that shared device has very high latency, especially when using screen mirror.
I was periodically testing device sharing and screen mirror, and the latency was much lower some time ago.
Initial investigation shows the latency is simply from WebRTC. In the same LAN network, it can reach 100ms.
Maybe I can add some UI to display information like available bandwidth and latency from https://developer.mozilla.org/en-US/docs/Web/API/RTCIceCandidatePairStats to help debugging and improve user experience.
Maybe some buffering and delayed acknowledgement and be deployed to optimize throughput in high latency situations.
(Optional) Browser console output
No response