w3ctag / design-reviews

W3C specs and API reviews
Creative Commons Zero v1.0 Universal
319 stars 55 forks source link

Background Blur API #826

Closed riju closed 1 year ago

riju commented 1 year ago

こんにちは TAG-さん!

I'm requesting a TAG review of Background Blur API.

On the Web, due to a lack of a standardized JS API for Background Blur and widespread demand (specially on Video conferencing/ Streaming use cases), developers have no options but to use ML frameworks like Tensorflow.js and other WASM libraries to satisfy their customers. This Background Blur API gives developers a choice to use the native platform's API.

Further details:

You should also know that...

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify riju

torgo commented 1 year ago

Hi @riju we're just reviewing in today's TAG breakout. This looks very good to us, particularly the use of transformstreams and multi-stakeholder participation in this work. You've noted that there's no security & privacy section in the spec yet - it would be good to see one, even if it's minimal.

torgo commented 1 year ago

Hi @riju one thing that came up in today's TAG call is the use of boolean instead of something more expressive such as percentage? Just taking a look at Whereby which we are running our TAG call in they already offer 2 blur modes - light and strong - so in this case it seems this API wouldn't be sufficient. We see in the explainer you're already considering something more extensible. We're worried about introducing an API that you'd have to change very soon. Is the group considering something more sophisticated than boolean? Thanks! ✨

riju commented 1 year ago

@torgo : Absolutely. This is one of the goals of Origin Trial We can learn if the API in its present form satisfies the needs of the users, or do we need something more to wow the users. One concrete example would be boolean vs enumeration. Right now, BG Blur only uses a bool which is an ON/OFF switch. Apple has a single Blur level today, but ChromeOS (5 blur levels) and Windows 11 (2 blur levels) offer more. We wanted to start off with something simple, get feedback and iterate on building something which the majority wants.

torgo commented 1 year ago

Hi @riju - Ok understood. As documented above, we already see examples in the market where a boolean would not be sufficient. Please note: the Whereby.com service I mentioned above is the one we're currently using for TAG calls. So just to be clear: our feedback is that the API looks good except for this issue. We strongly recommend you move to something more expressive than Boolean. Thanks!

matiaslopezd commented 1 year ago

Awesome API! But I have some doubts. I've tried it with various cameras (both new and old) and it only seems to work if the camera has a background blur feature?

I've been testing with this example: https://googlechrome.github.io/samples/image-capture/background-blur.html

We currently have many companies using virtual/blur backgrounds on the web, but the performance is pretty subpar, it even creates bottlenecks in the main thread, breaking other things, so having a native implementation would be fantastic!

riju commented 10 months ago

Awesome API! But I have some doubts. I've tried it with various cameras (both new and old) and it only seems to work if the camera has a background blur feature?

I've been testing with this example: https://googlechrome.github.io/samples/image-capture/background-blur.html

We currently have many companies using virtual/blur backgrounds on the web, but the performance is pretty subpar, it even creates bottlenecks in the main thread, breaking other things, so having a native implementation would be fantastic!

Hi @matiaslopezd , good to hear that. Yes, on Windows it works only with support for Windows Studio effects. On Mac, any M series will work and on ChromeOS, there's a list of relatively newer client devices, but the list is increasing.

Please join the Origin Trial and give us feedback.