search

Xharlie / pointnerf

Point-NeRF: Point-based Neural Radiance Fields
Other
1.1k stars 128 forks source link

Consideration of function "w2pers" in Neural Points module? #60

Open MobiusLqm opened 2 years ago

MobiusLqm commented 2 years ago

Hi, Charlie, thanks for your awesome work of PointNeRF.
But I had a question about the code In "w2pers" function of NeuralPoints.py. According to my understanding and your codes，"point_xyz" is the point position in reference view coordinates, which has shape(N,3), and "Campos" is the origin of target-view camera in reference view coordinates, "camrotc2w" is the transform matrix to transform points from target view coordinates to reference view coordinates(According to "getitem" function of dtu_dataset.py). So my question is:

  1. what's the meaning of point_xyz_shift?
  2. I think the second line of codes below have the same function of "Point_xyz_shift@camrotc2w", do you want to transform the point from reference view coords to target view coords? but point position is a row vector, why didn't you transpose the Matrix?
  3. what is the purpose of dividing x and y by z to get 'xper' and 'yper'?

`

point_xyz_shift = point_xyz[None, ...] - campos[:, None, :] # (1,N,3) xyz = torch.sum(camrotc2w[:, None, :, :] *point_xyz_shift[:, :, :, None], dim=-2) # same as point_xyz_shift@camrotc2w???

xper = xyz[:, :, 0] / xyz[:, :, 2] yper = xyz[:, :, 1] / xyz[:, :, 2]

` Looking forward to your reply! best regards

MobiusLqm commented 2 years ago

@Xharlie

Xharlie commented 2 years ago

hi we have developed everything in perspective coordinate first, so, if it is not used in the word coordinate, it might be a historical issue

SeaBird-Go commented 1 year ago

@MobiusLqm , Have you understood the function of w2pers? I still very confused about this function.

kaichen-z commented 1 year ago

Dear [MobiusLqm],

It's the same thing. I was confused in the first place. But considering the size of two tensors (B, 3, 3) and (B, 1, 3). The operation proposed by the author is correct.

Yi-yezi commented 12 months ago

hi,@SeaBird-Go @MobiusLqm , @kaichen-z Have you understood the function of w2pers?I think, point_xyz_shift = point_xyz[None, ...] - campos[:, None, :] xyz = torch.sum(camrotc2w[:, None, :, :]* point_xyz_shift[:, :, :, None],dim=-2) the first line want to transform xyz from world coordinate to camera coordinate, but only minus the campos is not enough, it should multiply a matrix w2c, and second line is not needed,because if xyz_shifted is in camera coordinate, it only need divide the Z value to transform to perspective coordinate, I am confused why it transform back to world coordinate I am not sure my think is right.