search

w3c / picture-in-picture

Picture-in-Picture (PiP)
https://w3c.github.io/picture-in-picture
Other
311 stars 42 forks source link

Canvas support #180

Closed namgk closed 4 years ago

namgk commented 4 years ago

Hi there,

Is there any plan to support PiP for canvas stream as well?

There are some use cases such as animations are drawn on a canvas and people want to see it while working on something else.

I'm thinking about feeding the canvas stream to a video element, however, the canvas might be animated by a requestAnimationFrame loop and when the tab is inactive, requestAnimationFrame is paused.

Thanks.

1valdis commented 4 years ago

I wanted to go this way too to make at least something interactive out of PiP video element, but seems like even streaming from canvas won't help if requestAnimationFrame is paused... Maybe use more stupid way with setInterval then? It shouldn't be paused, should it?

1valdis commented 4 years ago

I tried setInterval, but Chrome throttles it to 1 second for background. I also tried OffscreenCanvas with Worker (I heard that setInterval isn't throttled in a worker), but it works as expected for a few seconds and then starts hanging also. What the hell. Can we do something to smoothly animate canvas for PiP video element, please? Like, consider a tab with PiP to be "foreground"? By the way, I wish this works not on desktop, but on Android, as I have a usecase where I want PiP continuously drawn from canvas to be above other apps or home screen.

beaufortfrancois commented 4 years ago

I've played a bit with Picture-in-Picture and canvas at https://beaufortfrancois.github.io/sandbox/media/picture-in-picture-timer and it seems to be working fine for me with requestAnimationFrame. Can you share your playground page so that I can have a look?

1valdis commented 4 years ago

@beaufortfrancois I've made 3 demos: 1) using requestAnimationFrame (src) 2) using setInterval (src) 3) using setInterval, Worker and OffscreenCanvas (src)

The positive result is, as you said, only for requestAnimationFrame, and only for Desktop. (I'm observing a little hangs when using requestAnimationFrame, but most of the time it works smooth).

However, on Android, which is my main target for this API, all 3 demos hang as soon as I try to switch app: see screen recording.

beaufortfrancois commented 4 years ago

Picture-in-Picture Web API has still not launched on Android. It still has some issues IIRC. @mounirlamouri may tell you more.

1valdis commented 4 years ago

Yeah I got to enable it with a flag. And that would be OK for my use case; the problem is that it hangs though.

mounirlamouri commented 4 years ago

On Android, the API is available behind a flag at the moment but given that we didn't launch, it's possible that some bugs are still around.

beaufortfrancois commented 4 years ago

As this seems to be a Chrome issue, I'm going to close this. Feel free to open a Chromium bug at https://crbug.com/?q=component:Blink%3EMedia%3EPictureInPicture and we'll make sure it's triaged properly.

guest271314 commented 4 years ago

@1valdis ReadableStream can be used instead of requestAnimationFrame() https://github.com/guest271314/MediaFragmentRecorder/blob/imagecapture-audiocontext-readablestream-writablestream/MediaFragmentRecorder.html.