search

livekit / client-sdk-flutter

Flutter Client SDK for LiveKit
https://docs.livekit.io
Apache License 2.0
229 stars 109 forks source link

[bug] Multiple videos showing at the same time causes error and ui bug. #465

Open patrikheinonen opened 4 months ago

patrikheinonen commented 4 months ago

Describe the bug

When multiple VideoTrackRenderers are displayed at the same time on the screen we get an error:

Flutter was unable to create enough overlay surfaces. This is usually caused by too many platform views being displayed at once. You may experience incorrect rendering.

Which results in the participants names etc not showing and causes the videos to take the highest level in the z axis of the screen so if there would be menus etc they would open beneath the videos.

To Reproduce

Run the example app latest main on flutter web. Add multiple people to the call either by livekit client or adding more tracks in the _sortParticipants method where remote participants tracks are added to the userMediaTracks list.

Expected behavior

To not get any error and to have all ui elements to displayed.

Platform information

Flutter web. Chrome and all browsers.

3.16.7

jezell commented 4 months ago

https://github.com/livekit/client-sdk-flutter/issues/358

Logged here, but in the mean time there is a hack that the flutter team put in to get around the limit: Modify your index.html.


            let appRunner = await engineInitializer.initializeEngine({canvasKitMaximumSurfaces: 100});

Still a lot of limitations with this approach that would be addressed with a better fix in livekit itself (for instance blurs and filters won't work on the video layer), but it is better than crashing.

patrikheinonen commented 4 months ago

@jezell It seems that wont work as the programm will just throw out of memory if displaying like 40+ VideoTrackRenderers at the same time

Screenshot 2024-02-15 at 10 03 25

jezell commented 4 months ago

@patrikheinonen yeah it's a hack, but it's the best hack until the videos are rendered into the canvas directly. It's definitely super annoying that Flutter web doesn't have better built in support for cameras and videos.

jezell commented 4 months ago

@patrikheinonen I had created a PR to permanently solve this kind of issue in flutter itself. Idea might be useful to you if you don't mind forking flutter to fix it:

https://github.com/flutter/engine/pull/41562