Webcam live Features (Background Blurring, Smart Re-lighting, Video Sharpening, Switch Backgrounds)

[ArenaGrenade]

Description of the new feature / enhancement

This PowerToy would essentially see the creation of a virtual webcam object which when connected to would trigger the start of a pipeline from the chosen real webcam which is processed and passed on to the virtual webcam and then the requesting application.

The processing pipeline would involve the use of AI and Image Processing to provide the following features:

• Background Blurring
• Background Replacement
• Re-Lighting
• Video Sharpening

Other additional features such as basic color correction and more complex video upscaling can also be applied. All the above features would be offered with fine control for professionals and also simplified control for ease of use with new users.

This PowerToy would follow in the footsteps of Nvidia Broadcasting and hope to integrate a complete Voice and Video modification setup. (The complete vision. But we start with the simple image-based video modifications)

Scenario when this would be used?

Any stream/video call taken using a Windows device would be using this Power Tool. Google Meet and Microsoft Teams offer such services over the web, but there can be cases where the same is unavailable. Moderators on Microsoft Teams can block these features selectively. There are also other platforms where this is not readily available.

Supporting information

A simple way to handle this without PowerToys is to use third-party applications. Streaming to a fake webcam device using OBS and applying said transformations to input camera stream is possible. While this is possible, it is not straightforward for a beginner to set up. There are other direct third-party apps capable of doing this but are paid and involve the need to maintain yet another application.

Some external tools examples: FineCam (Paid, Free BG removal) XSplit VCam (Paid) ChromaCam (Paid, Some free features but heavily limited on FPS and quality)

None of the above offer a one-stop shop for all useful modifications. OBS gets the closest with the ability for fine control. But, it involves expertise and understanding of video processing and image processing. A simple no-brainer tool within PowerToys would be a great help.

For the best representation of the vision of this PowerToy, kindly refer to Nvidia Broadcasting Suite

[crutkas]

/feedbackhub

[microsoft-github-policy-service[bot]]

Hi! We've identified this issue that best is suited for Windows Feedback Hub. To do this on Windows, WinKey+F will bring it up!