[Feature Request] Unity has support for better window transparency

[Mornedil]

Currently, VSeeFace gets bright outlines around models while UI is active, and when the UI is hidden, those outlines are mostly removed with jagged edges around the model and poor anti-aliasing. I'm not sure how VSeeFace handles window transparency, but it seems like it's using a custom solution judging by the result.

This can be fixed!

Unity cameras have built-in support for window transparency with good quality anti-aliasing on the model. To enable this, in Unity's Camera component, all you have to do is to set the "Clear Flags" property to "Solid Color", and then set the Background color property to 0, 0, 0 and also the alpha to 0. With this setup, the unity window becomes transparent, and any rendered model gets perfectly anti-aliased outlines.

I have tested it in VSeeFace by importing a custom camera with a VSFAvatar model. Example Image (however VSeeFace currently ruins the anti-aliasing when hiding the UI)

If the built-in cameras of VSeeFace used these settings, we could get much better model outlines. Hope this helps!

[emilianavt]

Setting the background color to black will only work for premultiplied alpha, which only works with Spout2 to the best of my knowledge, not with Game Capture. The current setup is for non-premultiplied alpha and should not result in jagged edges or outlines as long as the UI is hidden.

[Mornedil]

I'm getting it to work in a game capture in Streamlabs right now, the example image from the original post is a direct screenshot of my Streamlabs output after adding those camera settings in Unity and importing that camera to VSeeFace, then using a game capture in Streamlabs to capture the VSeeFace window. The result ends up much smoother than when using the built-in VSeeFace cam so that's why I thought I'd mention it :)

Edit: Here's a higher resolution example screenshot from Streamlabs

[emilianavt]

The screenshots look normal to me. Can you show a screenshot that demonstrates the issue you otherwise have?

It might be related to this or a Streamlabs specific issue. Generally, I'd recommend to use OBS instead.

[Mornedil]

Ah yes, the issue described in that link is most likely it. I made a comparison screenshot to show what I'm experiencing in Streamlabs.

I suggest zooming in on the edges in the different images to more clearly see the difference in anti-aliasing.

• In image 1 (Default VSeeFace Camera, UI active), I get bright outlines.
• In Image 2 (Default Cam, UI hidden), The outlines are mostly gone, and the result looks pretty good, although the anti-aliasing isn't the best and there's a slight remnant of the bright outlines in some areas.
• In Image 3 (Custom Cam, UI active), I get no outlines, and the anti-aliasing looks perfectly smooth.
• In Image 4 (Custom Cam, UI hidden), the act of hiding the UI adds some jaggedness to the model's edges again, and there's a very thin outline based on the custom camera's background color.

I think the added outlines and jaggedness in Image 4 is due to VSeeFace's attempt at smoothing out the model's edges/outlines. I have noticed if I import a VSFAvatar with the custom camera already active (which is not recommended), the custom camera takes priority even with UI hidden and it keeps the smooth anti-aliasing.

Even if it's a Streamlabs-specific issue, I would still suggest implementing this camera method since it does look much smoother and fixes the outline issue for Streamlabs-users. It already seems to work with minimal changes - only some camera property changes for the software's main camera. Could also add a setting to enable/disable the different camera, in case the current camera is preferred by OBS-users.

[Mornedil]

Has this been considered since the suggestion? It's a quick fix by changing Unity camera settings to allow the Unity player to natively render a scene with a transparent background. I would suggest to at least test it out to see the difference.