PiP starts lagging when using a video background.

[vbabenko]

Chrome: v114 OS: MacOS Ventura 13.4.1 Video Background processor: https://www.npmjs.com/package/@twilio/video-processors

This video background processor is based on Google implementation.

These processors run TensorFlow Lite using MediaPipe Selfie Segmentation Landscape Model and requires WebAssembly SIMD support in order to achieve the best performance.

STR:
1. Start a video and apply the video background from tab1
2. Activate Picture and Picture functionality
3. Click on tab2 and check the video inside PiP

Actual result: video is extremely lagging
Expected result: video is playing

Note: When I deactivate the video background feature, the video inside PiP works fine again

[beaufortfrancois]

Can you share a test case so that we can reproduce locally to investigate what is going on?

[vbabenko]

Hi @beaufortfrancois, thank you for your quick response!

The easiest and fastest way to reproduce is:

1. Open this link https://twilio.github.io/twilio-video-processors.js/examples/virtualbackground/

   It will automatically start your cam (note, grant browser permissions if it's blocked)

2. Select any available video backgrounds
3. Open Chrome console and run this code snippet

   const pipWindow = await documentPictureInPicture.requestWindow();
   pipWindow.document.body.append(document.getElementById('video-input'));

   It will open PiP modal

4. Resize the modal to see yourself (its too small)
5. Open another tab in Chrome and check you Pip video

ACT: The video is lagging

[vbabenko]

Hi there, Any updates? Were you able to reproduce the issue? Thank you!

[vbabenko]

Hi, @beaufortfrancois any news?

[beaufortfrancois]

Sorry for the late response @vbabenko. I'm back from vacation and I can definitely reproduce the issue locally. I've just filed https://bugs.chromium.org/p/chromium/issues/detail?id=1465802 so that @steimelchrome and @liberato-at-chromium can have a look.

[steimelchrome]

Closing as the related chromium issue is marked as fixed