search

centreborelli / satnerf

Satellite Neural Radiance Fields
https://centreborelli.github.io/satnerf/
BSD 3-Clause "New" or "Revised" License
120 stars 22 forks source link

About the network input #20

Closed WBS111 closed 1 year ago

WBS111 commented 1 year ago

Why the network input does not need the view direction but only the sunlight direction. Theoretically, the color of the same point will be different when viewed from different viewpoints under the same sunlight direction. Is it reasonable to input only the sunlight direction and not the view direction?

rogermm14 commented 1 year ago

Hello @WBS111 ,

Thank you for your interest. Note that the viewing direction is always used implicitly since NeRF is based on shooting rays with a viewing direction and not just random points in 3D space. In our work we simply do not allow the viewing direction to directly explain color changes. As explained in the paper, this is because we assume a Lambertian reflectance model.

If a satellite scene is viewed simultaneously from different angles, we assume that the colors of non-occluded objects will vary little between views. However, if each image has a different sun angle, the colors of the same objects will change a lot (mainly because of shadows). This is why we prioritize the angle of the sun. In addition to that, small occasional color changes and global biases are managed by the model as well (see the beta and ambient color magnitudes discussed in the paper).

Best, Roger