Questions about the radiance field color loss

[YiningPeng]

Excellent work! Here is my question: Is the radiance field color (computing by volume rendering) uesd as the indirect lighting, same as the radiance field color used in the loss L_RF? If so, Why does the indirect lighting need to be consistent with GT color? I apologize for the possible error in my understanding. Looking forward to your answer and thank you for your excellent work!

[wonjunior]

I am not affiliated to or involved in the research of this repo but if that helps I can share my understanding of the method.

The radiance color obtained from vanilla TensoRF is used as guidance for learning the volume rendering and geometry of the scene (it benefits from a 10k steps warmup at which point the brdf rendering is not yet trained). After this point, both radiance and physics-based renderings are supervised eq 11 in parallel. Keep in mind l_PB loss is kept with a lower weight than l_RF. The motivation behind using radiance for 2nd bounce ray marching is that it's cheap to query the tensoRF. So in other words: radiance is not originally made for 2nd bounce, it is needed to train the brdf which is more ambiguous to train alone. They leverage the predicted radiance to obtain an efficient way to compute indirect light.

[Haian-Jin]

The radiance field constructed from the input GT images provides information for the indirect lighting, i.e. the indirect light from one particular direction observed at one particular position should be aligned with the radiance color observed. Therefore, directly querying the radiance field from the incident lighting direction and using the results as the indirect light from that direction can avoid the costly multi-bounce ray tracing computing process.