search

mozilla / standards-positions

https://mozilla.github.io/standards-positions/
Mozilla Public License 2.0
651 stars 72 forks source link

MediaTrackSupportedConstraints.backgroundBlur #658

Open riju opened 2 years ago

riju commented 2 years ago

Request for Mozilla Position on MediaTrackSupportedConstraints.backgroundBlur

Other information

The PR that introduced this behavior to the medicapture-extensions spec and the explainer was co-authored by Apple engineer Youenn Fablet and approved by Jan-Ivar Bruaroey (Mozilla).

Link to entry on the Chrome Platform Status: https://chromestatus.com/feature/5077577782263808

WebRTC meeting minutes: https://www.w3.org/2022/05/17-webrtc-minutes.html#t07

Explainer: https://github.com/riju/backgroundBlur/blob/main/explainer.md

bgrins commented 2 years ago

@riju will this only be implemented on platforms which have native API support, or will Blink maintain an implementation for cases where this isn't support?

bgrins commented 2 years ago

will this only be implemented on platforms which have native API support, or will Blink maintain an implementation for cases where this isn't support?

@riju just a ping on this question. If the latter, I'd like to hear more about the implementation plan - is this using an existing library for performing the blur or does this require new code and data to train any necessary models?

ttoivone commented 1 year ago

Being involved with the work, I'm familiar with the implementation. Our current implementation for Chromium only uses the existing Windows API KSCAMERA_EXTENDEDPROP_BACKGROUNDSEGMENTATION_BLUR. It does not implement background blur which is not available if the native API doesn't support it and we don't have a plan to do so.

riju commented 1 year ago

@bgrins : Apologies, for such a late reply. No use of any library and no training models, just use native platform API (drivers would encapsulate the models + inference). BG Blur or any effects that are not supported by the camera driver need to be done in software like the Meet effects pipeline or Breakout Box processing like in this example.

Here's the Intent to Prototype thread for Chrome. We have implemented the API for Chromium based browsers.

Here's a demo page if your would like to try out using Chrome Canary (using a flag --enable-blink-features=MediaCaptureBackgroundBlur)

We would plan an Origin Trial sometime soon on Chromium, so would like to hear Mozilla's position. @jan-ivar from Mozilla is the Editor of Mediacapture-extentions specification where this API resides and has been involved in the discussions.

bgrins commented 1 year ago

I suggest we mark our position as positive given that this is using existing platform APIs. Background blur is a popular feature and making it possible for sites to provide it without requiring third party libraries seems like a good thing.