search

google / hypernerf

Code for "HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields".
https://hypernerf.github.io
Apache License 2.0
896 stars 105 forks source link

Questions about camera coordinate and poses #3

Closed dogyoonlee closed 2 years ago

dogyoonlee commented 3 years ago

I really appreciate to your awesome work!!

I have some questions about the camera pose matrix in camera folder's json files.

I'm reimplementing your paper on pytorch based nerf code.

However, I have confused about camera coordinate in nerfies dataset.

  1. In load_llff.py in nerf pytorch code, they load the camera poses from given poses_bounds.npy file and correct their correct rotation matrix like follow codes.(poses matrix format is (3, 5, frame_num)

image

However, I could not find any correction part on your code when you generate the poses of dataset after loading them from given camera information json files.

I guess you used different coordinate system compared to original nerf(using llff dataset)

Then if I want to use your dataset as llff format and ray function, should I correct poses matrix like above code?

If you don't mind, could you explain the camera coordinate axis format of COLMAP results, NeRF(llff), and Nerfies(HyperNerf)??

  1. for dynamic scene dataset(called nvidia-dataset), there is no undistortion part for camera pose like follow part of your code.
image

I wonder the necessity of above part because nerfies dataset images seems to not be distorted. and the dynamic scene dataset is pre-undistorted image?

  1. Did you implement your code on NDC coordinate?? I wonder should I use the ndc_rays

I really hope your answer and appreciate to your great work again!!

Thank you.

keunhong commented 2 years ago

Answered in https://github.com/google/nerfies/issues/39.

Thanks!