Isabella98Liu
commented
10 months ago Hi Ziyi, thanks for your great suggestions! For evaluation metrics, I agree that decomposed materials (albedo/roughness/metallic) are important ground truth for inverse rendering evaluation. In fact, we provide OLAT images which can be used to generate pseudo GT albedos and normal maps through photometric stereo, and we also provide ground truth illumination in SGs format.
For the choice of objects, our primary focus has been on encompassing a wide variety of surface materials, while we haven't prioritized geometry complexity. To make sure all lights are well calibrated and consistent, we capture all the objects in a controlled dark room lab setting, thus eliminating any in-the-wild illumination in our dataset. The prospect of capturing in-the-wild data is an excellent consideration for our future endeavors. Thanks again for your insightful thoughts and valuable comments! :)
Hi, thank you for your outstanding contributions. Indeed, inverse rendering requires some dedicated real datasets to evaluate its effectiveness, and you have pioneered this direction.
I have some thoughts on inverse rendering.
About the evaluation metrics: I believe that rendering tasks, such as novel-view synthesis and PBR rendering, are important metrics for assessing inverse rendering, reflecting the fidelity of the reconstruction. However, I think a more crucial metric for inverse rendering is the accuracy in decomposing materials (albedo/roughness/metallic), environment lighting, normals, and visibility. I've noticed that, although TensoIR has good rendering metrics, the decomposition of its individual components isn't perfect. It tends to bake shadows and indirect light into the albedo. It would be great if the dataset could include ground truths for albedo, roughness, and environment light (though I'm unsure how to obtain the real dataset's ground truth for albedo).
About the choice of objects in the dataset: I believe that the objects in the current method are mostly simpler single-surface objects. They don't have the complexity of self-occlusion and outdoor shadows like NeRF's hotdog/lego. I think if some more complex objects could be incorporated into the next work, it would provide a more scientific evaluation of the robustness of inverse rendering methods.
Here are the experimental results of TensoIR on some complex objects. The inherent challenge is that the method struggles with shadows caused by self-occlusion and reflections from indirect light. TensoIR, due to the lack of extensive priors on the decomposed components, tends to make the albedo and rendered color as close as possible.
