search

keijiro / NNCam

Virtual background with Unity Barracuda
Apache License 2.0
249 stars 35 forks source link

Linear Color Space #14

Open matijamandurov opened 2 years ago

matijamandurov commented 2 years ago

Does the project support the Linear Color Space option, background removal seems to be greatly reduced in quality when switching from Gamma to Linear Color Space?

keijiro commented 2 years ago

Maybe you have to add linear-to-gamma conversion to the preprocess compute kernel.

matijamandurov commented 2 years ago

Perfect, thank you. Do you have a quick example of how I could add that. I assume I have to convert the rgb component using a method similar to this, only in the other direction?

http://chilliant.blogspot.com/2012/08/srgb-approximations-for-hlsl.html?m=1

keijiro commented 2 years ago

Yes.

matijamandurov commented 2 years ago

Thank you, so I tried adding the above formula to both the Preprocess Compute Shader and the Compositor Compute Shader but there seems to be some issues with it. The closest I have come to getting the same quality of the cutout was if I added the conversion twice in the Compositor. Do you by any chance know exactly where in the flow should I be converting from Gamma to Linear?

keijiro commented 2 years ago

Maybe here: https://github.com/keijiro/NNCam/blob/main/Assets/NNCam/Preprocess.compute#L16

SilverXenolupus commented 2 years ago

After struggling with this one for a while, I found that if you change line 84 of SegmentationFilter.cs to var segsRT = output.ToRenderTexture(RenderTextureFormat.DefaultHDR, 0, 0, 1.0f / 32, 0.5f);, the lost data is conserved. The problem was the RenderTexture was losing data outside the 0 and 1 range that the Linear colorspace is not confined to, so the RenderTexture must be HDR. No other changes are necessary.

matijamandurov commented 2 years ago

perfect. Works just as expected. Consider this resolved and maybe should be added to the project