Support "Auto Picture-in-Picture"

[beaufortfrancois]

Video meetings web apps would benefit from automatic Picture-in-Picture behavior when user switches back and forth between web app and other applications/tabs. This is currently not possible with the user gesture requirement in https://wicg.github.io/picture-in-picture/#request-pip (step 6).

After some discussion, we think a declarative behaviour may be the best we can can propose as it allows browsers to correctly implement an “auto-pip on visibility change” feature without opening up a security or annoyance can of worms. Here's what I'm envisioning.

Note: I’m using the autopictureinpicture keyword name. We can use comments to be pick a better name later if needed.

IDL

partial interface HTMLVideoElement {
  attribute boolean autoPictureInPicture;
};

Javascript

const video = document.createElement('video');
video.srcObject = await navigator.mediaDevices.getUserMedia({ video: true });
video.autoPictureInPicture = true; // <- This is where the magic happens.

video.addEventListener('enterpictureinpicture', function(event) {
  // Video entered Picture-in-Picture (potentially automatically)
});
video.addEventListener('leavepictureinpicture', function(event) {
  // Video left Picture-in-Picture (potentially automatically)
});

When autoPictureInPicture is set to true, video will attempt to enter Picture-in-Picture automatically (without user gesture) when page becomes hidden. And existing Picture-in-Picture window will be closed automatically when page becomes visible. We'll use Page Visibility API events for this.

I believe this proposal is simple and concise but there is something to consider. What if we have two video elements with autoPictureInPicture (or more). Spec could say that the first video element matching document.querySelector('video[autopictureinpicture]') will always get the “auto-pip on visibility change” behaviour. But what if the video element is not attached to the DOM? In that case, we would have to monitor all autoPictureInPicture values set, potentially managing a stack of video elements allowed to use this feature. Eventually, this would have to be defined in the spec. In the end, there can be only one.

Note that I’ve considered alternatives such as a document.autoPictureInPictureElement attribute that web developers could set. However I’d prefer using a HTML content attribute as this doesn't require web developers to use Javascript for such a simple opt-in thing. And it also pollutes the document element.

What do you think?

[jernoble]

Generally, we (WebKit) are going to be constrained by platform behavior, so I would prefer if the language around "when" and "what" gets auto-pip'd be a little vague. Just as an example, paused media doesn't get auto-pip'd on iPad; but I'd understand if different UAs on different platforms made different choices there.

All that said, I like this approach.

[beaufortfrancois]

My WIP PR is available at https://github.com/WICG/picture-in-picture/pull/112. Please let me know if that still looks good to you.

[trlkly]

How do you deal with the annoyance factor of sites that wish to abuse this feature? A video that stays on screen even if you change tabs is an advertisers dream. Websites already exploit the ability to put the video on top of text when the user scrolls down, which tends to only annoy users.

It seems like this should be a feature that the user has to opt into on a per site basis.

And, yes, I am aware this is only for UWP for now, but the wording very much suggests that you plan to move this to full web use.

Apologies if this is the wrong place to ask. It's hard to find where I can ask about this sort of thing. I personally would never want this to automatically happen, and would prefer to explicitly enable PiP when I want it.

[mounirlamouri]

And, yes, I am aware this is only for UWP for now, but the wording very much suggests that you plan to move this to full web use.

The spec wording is meant to leave the choice to the implementation but for the reasons you gave, I would be surprised that a browser ships this for the drive-by web without very strong limitations. Chrome has no such plans at the moment.

[beaufortfrancois]

@trlkly Thanks for your feedback. Which wording made you think we would move this to full web use?

[pshem]

@beaufortfrancois The current implementations, for example YouTube's "inter tab" PIP do exactly this - they keep playing a video, whose tab you just closed, if you end up in another YT tab. The standard should prevent this from happening when ending up in a tab from a different domain.

[jonnymaceachern]

How do you deal with the annoyance factor of sites that wish to abuse this feature?

Maybe not the best solution, but it could be handled the same way notification permissions are handled.

[trlkly]

That would suggest that the default is the prompt the user if a site tries to activate picture-in-picture mode, with an option to deny and "never ask again, along with an option in the settings to disable it for all sites (which I would use, since I can manually activate PIP if I want it.)

What I'm wary of are site developers trying the whole "user interaction" option, which just does not work. I've seen that used to try and stop autoplaying video, and the result is that they just find a way to create an "interaction" that allows autoplaying video.

Resist that. Make it prompt. Yes, even if they click a button that's part of the page. Because then they'll make it where something you would reasonably click on gives them permission to activate PIP. Any reasonable way to bypass a prompt will be exploited.

Heck, if it's like other prompts, they'll even show pictures that try to tell the user that they need to click "Allow" so the page will work.