Artifacts with depth maps?

[Jerrypiglet]

Hi there I noticed some potential issues with a few depth maps, where pixels that belong to a open window have large depth values, instead of 0. Here is the 0003 image of the released bedroom scene.

Note the yellow strip in the center figure, which are inside the white area of the 'window hole'; the pixels have large depth values but ideally should be infinite (0) because they correspond to the outdoor env. I pick two locations (red dot and green dot) on the figures, and query the depths, getting 4.3840003 3.1920002 respectively, but the former one should be 0.

As a result, if you simply back-project depth to 3D, you get fantom geometry (in red circle).

Not sure if this is an issue with the way I read the depth file (cv2.imread(filename, -1)), or the depth file itself.

[jingsenzhu]

Thanks for your careful observation! This is because our original scene in fact is a model of a large house containing several rooms. The "yellow strip" in the depth map you observed is the outer wall of another room, which happens to be seen by some cameras😂

[jingsenzhu]

From my point of view, these observations of "other rooms out of the window" should happen rarely. I think this will cause negligible impacts.

[Jerrypiglet]

Oh I see. Thanks for the timely explanation! Yeah geometries that lie outside of the room do not exactly following the inverse rendering assumption and lighting model, that light transport only happens between objects/emitters inside the room plus received light from outside envmap through a window, so there is a chance that those pixels yield larger re-rendering errors. But I agree that given the number of those pixels is small compared to all pixels, this might not be a big issue.