Picture-in-Picture

[beaufortfrancois]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: Picture-in-Picture
• Specification or proposal URL: http://wicg.github.io/picture-in-picture
• Caniuse.com URL (optional):
• Bugzilla URL (optional):

Other information

This specification intends to provide APIs to allow websites to create a floating video window always on top of other windows so that users may continue consuming media while they interact with other content sites, or applications on their device.

• Chromestatus URL: https://www.chromestatus.com/feature/5729206566649856

cc @mounirlamouri

[beaufortfrancois]

For info, Picture-in-Picture got a LGTM at https://github.com/w3ctag/design-reviews/issues/226#issuecomment-372718715

[beaufortfrancois]

We've announced recently our intent to experiment with Picture-in-Picture in Chrome. See https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/xQiDYZEnwaA/MNGbYJnaBwAJ

[jetvillegas]

I spoke with a number of Mozilla folks who work in this area, including the folks who implemented the min-vid Firefox browser extension.

We're not opposed to the proposal, and agree that the use cases are valid, but we don't have bandwidth to implement or send detailed spec comments at this time. Please keep us posted on your implementation experience, and we'll take a closer look as engineers free up from other projects.

[beaufortfrancois]

FYI Picture-in-Picture has shipped in Chrome 70 for Linux, Mac, and Windows. https://developers.google.com/web/updates/2018/10/watch-video-using-picture-in-picture

[beaufortfrancois]

For info, we're currently thinking about the next iteration of this API for arbitrary content. Is that something Mozilla would be interested as well?

[martinthomson]

@mikeconley had some great thoughts on this and might be able to offer a view.

[mikeconley]

Sorry for the delay in responding. I'm actually going to redirect a request for comment to @marcoscaceres.

[beaufortfrancois]

@marcoscaceres (gentle ping)

[marcoscaceres]

reading it now... sorry for the delay y'all.

[beaufortfrancois]

@marcoscaceres Thanks for the spec feedback! We'd be interested in your thoughts as well in the V2 proposal for arbitrary content: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/uK0hyACy_fg/JVXGUVylAAAJ as noted in https://github.com/mozilla/standards-positions/issues/72#issuecomment-493096734

[marcoscaceres]

Tl;dr: I think we should "defer". However, if we want to ever implement a PiP capability in Firefox, we should do some serious UX prototyping and see if we can do this without needing an API. If it proves too challenging/impractical, we should definitely prototype the API. We could even start by just enabling the OS-level PiP in MacOS, to match Chrome and Safari (no API needed!) just to see if people would use it.

The long version: Having read the V1 spec... the spec is well thought out, but has a couple of little potential race conditions and underspecified things - but nothing that can't be easily fixed. The API is straight forward, and integrates well with various new security features afforded by the platform, like feature policy.

There is, nevertheless, a larger question if we need this API at all: Firstly, the use case for this capability (PiP) is undeniably useful - I personally used this capability a lot, both on my iPad and on my Mac. I've live-tweeted live events using PiP, and I've used PiP while watching educational videos, where I needed to see the video while I was coding a long to something... so I'm sure I'm not alone in seeing PiP's value.

Now, for the API, however, I've been able to use PiP in Safari without the need of API at all which really begs the question at to why we need this API at all.

On YouTube, using PiP in Safari and in Chrome, it's somewhat annoying for end-users: you need to right-click twice to get the native PiP option to appear. Having the API would allow PiP to be integrated into the custom right click menu (the spec also shows examples of this).

On an iPad, Safari overcomes the issues around UI by only making the PiP button available while in full screen (which only shows native video controls)... this is somewhat of a limitation.

So, from this - we should discuss with, as a I said above, I think we (Firefox) should discuss seriously with our UX folks what we can do here... and if we do go down the API route, we should go back and review if we need to balance out developer control over this feature with user's control over this capability (e.g., why should the site never allow a user to not do PiP for any video?).

[andreasbovens]

if we want to ever implement a PiP capability in Firefox, we should do some serious UX prototyping and see if we can do this without needing an API.

Update: we've now shipped user-invoked PiP on Firefox for Windows, macOS & Linux.

[marcoscaceres]

Ok, I guess we let this cook for a year or so, and then re-evaluate if we need the API.

[enjikaka]

How about we stop cooking that right away? 🙈

I've just implemented PiP on https://listen.tidal.com. Being released next week.

Here the custom implementation from Firefox over the API proposal from Google that Chrome and Safari (Tech Preview) implements causes issues.

• We cannot hide the icon
• We cannot let users enter PiP through our own icon
• The icon disappears randomly when switching between our different views (footer player <-> now playing <-> full screen). (The

Could you please use the same API instead? :)

Screenshots for context: https://imgur.com/a/ahDhjuR

[joaoBeno]

How about we stop cooking that right away? 🙈

I've just implemented PiP on https://listen.tidal.com. Being released next week.

Here the custom implementation from Firefox over the API proposal from Google that Chrome and Safari (Tech Preview) implements causes issues.

• We cannot hide the icon
• We cannot let users enter PiP through our own icon
• The icon disappears randomly when switching between our different views (footer player <-> now playing <-> full screen). (The element is moved to its new position here)

Could you please use the same API instead? :)

Screenshots for context: https://imgur.com/a/ahDhjuR

Lol... So Firefox, should just rubber stamp what Google Blink does and let them rule? The API is not approved, and Firefox is testing an opposing implementation that is better for the end user... My computer, my browser, my network, it surely should be my rules...

That's why I picked Firefox as my browser since Opera became Chrome...

[enjikaka]

I'd be all for Mozillas solution if it was better than the other. But this is the worst. See my points on interop within existing apps, which they can not do properly with this approach.

Den mån 10 feb. 2020 17:24joaoBeno notifications@github.com skrev:

How about we stop cooking that right away? 🙈

I've just implemented PiP on https://listen.tidal.com. Being released next week.

Here the custom implementation from Firefox over the API proposal from Google that Chrome and Safari (Tech Preview) https://webkit.org/blog/9672/release-notes-for-safari-technology-preview-97/ implements causes issues.

• We cannot hide the icon
• We cannot let users enter PiP through our own icon
• The icon disappears randomly when switching between our different views (footer player <-> now playing <-> full screen). (The element is moved to its new position here)

Could you please use the same API instead? :)

Screenshots for context: https://imgur.com/a/ahDhjuR

Lol... So Firefox, should just rubber stamp what Google Blink does and let them rule? The API is not approved, and Firefox is testing an opposing implementation that is better for the end user... My computer, my browser, my network, it surely should be my rules...

That's why I picked Firefox as my browser since Opera became Chrome...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mozilla/standards-positions/issues/72?email_source=notifications&email_token=AAGWHD5Z4HSHDZMPSW2QWL3RCF5SHA5CNFSM4EVKEKUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELJEQXA#issuecomment-584206428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGWHD27A4DRKGE5IZGXBBTRCF5SHANCNFSM4EVKEKUA .

[ric2b]

You can simply hide the icon when the browser doesn't support the API. Firefox users will know how to enable PIP. That fixes the dead button issue.

And honestly I prefer that my browser has a standard way to enable PIP that actually works on all websites. Imagine if there was a "back button" API so that websites could style it/etc, it would be awful. I don't think this is very different from that.

[enjikaka]

I can hide my custom button, yes. But there is no way for me to hide the PiP button from Firefox or the PiP graphic in

This needs a proper API. Mozilla should either implement the proposal from Google (buhu they didn't come up with it first 😢), or create something similar so applications can control how this is entered, left and displayed.

You don't throw stuff in and break UIs as a browser. Very bad practice.

Den mån 10 feb. 2020 23:36Ricardo Amendoeira notifications@github.com skrev:

You can simply hide the icon when the browser doesn't support the API. Firefox users will know how to enable PIP. That fixes the dead button issue.

And honestly I prefer that my browser has a standard way to enable PIP that actually works on all websites. Imagine if there was a "back button" API so that websites could style it/etc, it would be awful. I don't think this is very different from that.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mozilla/standards-positions/issues/72?email_source=notifications&email_token=AAGWHDYNWLUMBNA5XJNNNNDRCHJGDA5CNFSM4EVKEKUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELKR2TQ#issuecomment-584392014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGWHD5TLC4GOHXU537WN6LRCHJGDANCNFSM4EVKEKUA .

[adamroach]

Thanks for the input. To be clear, this issue has to do with Mozilla's support of the API from a standards perspective. There is a separate project that tracks the implementation status of platform features.

I'd also like to remind people to look over the Community Participation Guidelines, and to keep their feedback polite and professional.

[marcoscaceres]

I've locked this thread until things cool down a bit.

@enjikaka, we really appreciate your input, and understand your frustration. We are trying some new things and they may not work out (with the UI we've been experimenting with). This is not an us vs them thing - but something we'd still to explore. We are constantly re-evaluating our assumptions, and having you show us how our UI doesn't work on your service is a helpful data point. However, statements like "buhu they didn't come up with it first" don't help make your case, and act as a disincentive.

[JeremiePat]

Hi!

Here's a quick feedback about that new feature in Firefox.

First of all, from a user perspective, thanks for bringing that awesome feature. It was something missing and it's great to have it available in Firefox at last.

That said, from an author perspective, I have to say that I'm quite confused.

First of all, I do not understand the heuristic to display the PiP UI. On a simple local test page with a single <video> element, the UI never show up where it appears on any video web site when overing a <video> element. So, first discovery: I cannot predict when that UI will show up to my users! 😥

Second, the current blue "native" UI widget is an excruciating scare on many UX.

The trick is that it's possible to somewhat work around that issue by playing the video within a canvas element 🙄. It's really unnecessarily time consuming but that's not the biggest problem. The true problem is because there is no API, authors can hide that questionable UI if they really want to, but in exchange, they cannot expose that feature back to their users in their own terms.

I could understand not being able to customize the PiP window content (even if it's not 100% satisfactory), but not being able to customize the UI to access it, that is a hard no go for me. What if the whole UI for the <video> element was not customizable? There is no good reason to stop authors for customizing a part of that UI, which is currently the case with PiP. I don't really understand the reasoning to not expose at least a minimal API to let authors create their own UI on top of that feature (at least as minimal as the requestFullscreen API).

So basically, the feature is here (and the user in me say thanks again) but it cannot be managed to suits authors expectation, which is just another reason to, sadly, dismiss Firefox.

[enjikaka]

It is also weird that a custom control from a browser vendor does not end up in the <video>-native controls, and also does not listen to the controls attribute. This whole implementation is badly planned from start to finish.

Even when the attribute is absent, however, user agents may provide controls to affect playback of the media resource (e.g. play, pause, seeking, and volume controls), but such features should not interfere with the page's normal rendering. For example, such features could be exposed in the media element's context menu.

https://www.w3.org/TR/2011/WD-html5-20110113/video.html#user-interface

I'd say that Firefox custom PiP button very much interferes.

Mozilla is clearly not considerate of developers. I'll stop using, recommending and caring about making the stuff I develop work in FF from now on.

[marcoscaceres]

@enjikaka we understand you are frustrated, but insulting the work of Mozilla developers won't help advance your case. Kindly ask that you follow our participation guidelines and be respectful. No one is saying we won't implement the API - we are just not doing it right now. Even if we agreed to implement the API, it could take us a while to get to it.

[enjikaka]

@marcoscaceres If you are not going to implement it right now, at least swiftly remove this bad alternative you've made up. You're welcome to add a custom PiP-button to the native video controls or the context menu, as per the spec.

[marcoscaceres]

@enjikaka, respectfully, it's our browser: We don't walk into your home and start telling you how to decorate or what you should eat. We really appreciate that you've brought issues to our attention that may negatively be impacting users and developers, but please refrain from telling us what to do.

[beaufortfrancois]

Do you think it would make sense to add partial support for the Picture-in-Picture Web API by simply supporting the disablepictureinpicture HTMLVideoElement attribute at https://w3c.github.io/picture-in-picture/#disable-pip

This way, the browser PiP button would be hidden for a video element if web developers set its disablepictureinpicture attribute to true. This would be as easy as below to provide a nice user/developer experience.

if (!document.pictureInPictureEnabled) {
  // Picture-in-Picture Web API doesn't seem to be supported.
  // Let's add a hint to the browser that a PiP browser button is not suitable.
  document.querySelector('#myVideo').setAttribute('disablepictureinpicture', true);
} else {
  // It's up to web developers to allow or disallow PiP at this point.
}

[JeremiePat]

Hey folks, I think everybody should cool down a bit here.

@marcoscaceres I totally agree with you that Mozilla takes whatever decision regarding their product. However, knowing Mozilla process reasonably well, I'm a bit surprise that, in its current state of implementation, that feature ride the train out of nightly so fast. Nightly could have been used to get the feedback regarding that feature in a way that would have been more peaceful. It's true that the prose of @enjikaka is, let's say, "spicy" in a way that is not really helping but I also understand his frustration very well: As a web author, the decision made by Mozilla to ship a feature that could be easily qualified as "half finished" and that introduce hard UX discrepancies directly within my carefully craft web content is infuriating. I urgently need to workaround a tricky unwanted UX change which has a direct impact in my own product roadmap, forcing me to postpone other features and therefor impact my business. In such a context I think it's reasonable to "let it go" when people start to be a bit harsh on comment.

So yes, as web author, we do not have to tell Mozilla what to do, but do not expect any thanks if you screw up our own product roadmaps and don't be surprise if we, web authors, just decide to give up on a browser that jeopardize our own business. To conclude, even if it's very important to have civilized discussion about web features, I'm not sure that Mozilla is currently in a position to lecture web authors when Mozilla is taking decision that impact their business which, very understandably, will piss them off.

[JeremiePat]

@beaufortfrancois

Do you think it would make sense to add partial support for the Picture-in-Picture Web API by simply supporting the disablepictureinpicture HTMLVideoElement attribute at https://w3c.github.io/picture-in-picture/#disable-pip

I haven't dig deep within the spec, but could you remind what is the reasoning to introduce a disablepictureinpicture attribute instead of having the feature just being toggle like the rest of the <video> controls with the controls attribute`?

From my web author perspective having PiP reacting to the controls attribute would be good enough as I'm usually disabling all controls to build them up back (and the whole flame in that topic is clearly about the UI issue rather that the whole feature itself). Regarding the current Mozilla implementation I would be satisfied if they would be okay to have the PiP UI honoring the controls attribute.

[enjikaka]

@marcoscaceres respectfully, it's our spec. A carpender doesn't walk into your home adding blue boxes to the right hand side of all windows on a whim. He follows regulations and a spec he too.

Here's the participation guidelines for making a browser. Sound like you need to read up!

[mikeconley]

If it helps at all, we've added the ability for us to reposition the toggle for sites on a particular domain. If the toggle is interfering with a piece of UI (for example, if it's intersecting a semi-transparent button and intercepting user clicks unexpectedly), we can position the toggle at different places along the right-edge of the video.

In the event that you'd like us to evaluate repositioning the toggle on your site, please file a bug on Bugzilla blocking this metabug. Alternatively, you can email me at mconley at mozilla dot com with a link to the site for us to test, and I'll file the bug for you.

[enjikaka]

@mikeconley Any position for the PiP button within <video> breaks the UI on https://listen.tidal.com in the footer player as there is a custom hover control there to open an alternative player view:

[mikeconley]

This looks rather like a bug - we attempt to hide the toggle on videos that are smaller than certain dimensions. This video should qualify for that exemption, but it's not. I've filed this bug for it.

We should move discussion of that issue to that bug, and try to keep this GitHub issue focused on the proposed PiP WebAPI, and Mozilla's position on it.

[jimmywarting]

Just discovered pip was available in FF (chrome user here) recently read How we built Picture-in-Picture in Firefox Desktop - (grate article btw)

• From one perspective I like that you have made pip available for all the videos out there but as a developer I would like to style/control it myself.
• I think a better approach would have been to develop something like the chrome pip extension that detects videos and lives outside of the DOM instead. and interacting with the OS instead like the touch bar for example.

It's sad to see 3 different implementation that all behaves differently. I understand it's good to be competitive and design new things to come up with something better

but i hope later that all can agree on one unify spec:ed version later - (whichever it may be)

---

I would not mind either if you kept it like the way it's right now. But if i detect that it looks bad on my website I would like to disable it and reimplement it myself - hence why i think a API would be useful

One thing i like about chrome + safari's implementation is that i can first detect when it enter/leaves the pip mode so i can style the wrapper element and have it look the same. for instance i set the video poster as a background (still have the video element on top with a opacity: 0 so they can still control it with the context menu also.) and i can let the custom control panel be visible at all time

(Partial support for the Picture-in-Picture Web API don't sound that bad either)

Google Dev blog had a good usecase also: Show canvas element in Picture-in-Picture window

const video = document.createElement('video');
video.muted = true;
video.srcObject = canvas.captureStream();
video.play();

// Later on, video.requestPictureInPicture();

what if it's a offscreen video element and the only visible is the canvas element? then there is no way to enter pip mode. unless you switch out the canvas element for the video element instead.

[mikeconley]

This looks rather like a bug - we attempt to hide the toggle on videos that are smaller than certain dimensions. This video should qualify for that exemption, but it's not. I've filed this bug for it.

Following up here, this bug has been fixed in the latest Firefox Nightly, and should ride out in Firefox 77 (should release June 2nd).

[theSdev]

I see the problem has to do more with the UI of the button than the lack of api, and saw a couple of suggestions to fix this in this issue, and am surprised that nobody has suggested a psuedo element for styling the button (likes the ones form controls have). Can Mozilla evaluate this approach?

[graywolf]

@theSdev wouldn't developers just style it as display: none or something and call it a day?

From the user perspective, I love how this is available everywhere and I don't have to hope that developers don't disable it for whatever idiotic reason.

The blue button can be hidden, there is option for that in settings so users that do mind it being there can just turn it off no?

[jculvs]

Any updates to this? I am seeing that the PiP does not show up for videos smaller than 45 seconds or so. Is there possibly just a way to change this threshold?

[mikeconley]

@jculvs If you set media.videocontrols.picture-in-picture.video-toggle.always-show to true in about:config, the toggle will skip most heuristics and always show.

[bkniffler]

This is not directly related to the UI issues that may be caused by PiP as discussed before, but more general PiP.

Whats the reason for FireFox (and all other major browsers) to implement PiP in a custom way? All OS now have native PiP capabilities that are streamlined perfectly into the OS experience.

Advantages of native PiP as seen in MacOS Safari:

• Low CPU/GPU costs (compared to FireFox)
• Works across workspaces and even over fullscreen applications
• No need to maintain custom code
• Consistent experience with other PiP on the same OS

Advantages of custom PiP

• Only one video at a time
• Limited possibilities for positioning (corners)
• Consistent experience across different OS

I greatly prefer the native experience, but this might of course be a personal preferences. Whats the opinion of others around here?

[jculvs]

@mikeconley I understand there is a way to adjust this as a user but I am serving video that I wish to disable PiP on. Most videos I serve are shorter than 45 seconds but a few go over that default threshold and if the users goes into PiP it breaks other features on my page. Most browsers allow me to disable this easily, is there any option like this on firefox as of now?

[mikeconley]

@jculvs No, there is not a way for sites to control whether or not the toggle appears.

[jonnyfux]

Any updates on implementing the API? I would really love to use requestPictureInPicture() method in Firefox.

I think there are a lot of use cases where it makes sense to support this feature, to just give my example: We offer a website with a live stream and other components with more information. If the API would be supported, the user could "take" the live stream with them while browsing through the site. The PiP mode would be the perfect solution for this :)

[cyfung1031]

I don't understand why FireFox provides a div.pip-small.clickable for users to do PIP but don't provide the related APIs.

Any reason to not giving PIP control to the developer?

In 2 years ago you it is called experimental, but now it should be confirmed as stable, right?

[mantou132]

I have an extension using requestPictureInPicture, but after waiting for many years, Firefox still has not implemented the API.

I don't understand why Mozilla does not implement the API, and even inserts a very annoying(developer could not disable it before) PiP button on the video.