Haian-Jin / TensoIR

[CVPR 2023] TensoIR: Tensorial Inverse Rendering
https://haian-jin.github.io/TensoIR/
MIT License
227 stars 12 forks source link

Doubts about the failure of the environment map estimation #17

Closed YiningPeng closed 11 months ago

YiningPeng commented 11 months ago

Hello, thank you for your excellent work. I would like to add the SDF field to your work for further reconstruction. I used the geometric representation of neus while retaining the density tensor. I tried it on the 'hotdog' scene of the Blender dataset. (1) The estimation of the environment map was found to be unreasonable, and the estimated albedo was severely darkened. Here is the environment map. YTSG8IEP2UQN3GBGT{9ELWY

(2) Besides, the estimated RGB image (nvs with radiance field) seems to have grids, which is not smooth and continuous. P_L3XR2Q5N%E)Y2L{ V8383

Could you give me some suggestions about it? Any reply will be appreciated.

Haian-Jin commented 8 months ago

Hi,

sorry for my late reply.

By observing the depth map of your uploaded image, It seems that the network treats the white backgound as part of the object. This may explain the failture case of your environment map and albedo.

A better debugging way is first adding SDF's way of representation(like NeuS as you used) to TensoRF instead of TensoIR. You can export depth map of TensoRF to check if it works. (if you can export a reasonbale mesh from your implementation of TensoRF+NeuS.)