search

NVIDIAGameWorks / kaolin-wisp

NVIDIA Kaolin Wisp is a PyTorch library powered by NVIDIA Kaolin Core to work with neural fields (including NeRFs, NGLOD, instant-ngp and VQAD).
Other
1.45k stars 133 forks source link

Render Color Equation #143

Open saptarshi1234 opened 1 year ago

saptarshi1234 commented 1 year ago

https://github.com/NVIDIAGameWorks/kaolin-wisp/blob/cb47e10f376e5ac8b6965c650d8a6b85b9bc968e/wisp/tracers/packed_rf_tracer.py#L148-L153

I have a doubt in this section of the code. When the background is white what is the interpretation of adding "ray_colors" to "(1-alpha)"? Will this not make the value of "color" to more than 1?

orperel commented 1 year ago

White background assumes the image was created with premultiplied alphas: https://microsoft.github.io/Win2D/WinUI3/html/PremultipliedAlpha.htm

Specifically: result = source.RGB + (dest.RGB * (1 - source.A)) but here dest.RGB is simply 1.0 due to white background.

This is only true if the data is compatible, and indeed the NeRFSyntheticDataset applies this fix, see here: https://github.com/NVIDIAGameWorks/kaolin-wisp/blob/main/wisp/datasets/formats/nerf_standard_dataset.py#L426-L428

Are you seeing any bug here? Thinking out loud here - maybe it makes sense to separate this param to actual "bg color" and "is_premultiplied_alphas"