Wrong setting with depth in PtsManipulator

[xiat0616]

Hi, there seems to be a bug in the code. In the function PtsManipulator, there are lines:

        projected_coors = self.xyzs * pts3D
        projected_coors[:, -1, :] = 1

and

        sampler = torch.cat((xy_proj[:, 0:2, :] / -zs, xy_proj[:, 2:3, :]), 1)
        sampler[mask.repeat(1, 3, 1)] = -10

However, by doing this, you are treating depth as the last element of homogenous coordinate, [x*d1, y*d1, d1] is equal to [x*d2, y*d2, d2].

As a result, this function do not care what is the depth, it basically treats every pixel as they are in the same plane.

Even putting random noise (as long as positive) as depth map, the rendered RGBs will be the same as the ones using GT depth maps. The rendered RGBs only depend on T, R, and the input RGBs.

Can you check and fix it? Thank you.