search

jasonyzhang / RayDiffusion

Code for "Cameras as Rays"
MIT License
525 stars 24 forks source link

Convention about Camera and Ray #8

Closed jytime closed 8 months ago

jytime commented 8 months ago

Hi Jason,

Thanks for the work! I am interested in trying "CAMERAS AS RAYS" in our ongoing project. After quickly checking the codes, it seems we can use cameras_to_rays to convert pytorch3d cameras to PxP rays, and rays_to_cameras_homography to get the cameras back (with intrinsics). I was wondering:

  1. Should we use cameras_to_rays and rays_to_cameras_homography for training? Should we keep the default flags, e.g., use_plucker=True?
  2. All the cameras are assumed to be NDC Pytorch3D camreas?

Any clarification is much appreciated!

Best, Jianyuan

jasonyzhang commented 8 months ago

Hi Jianyuan!

Yes, default flags should be good. We always train with a 16x16 set of rays with Plucker coordinates. Just a heads up, when converting rays to cameras using rays_to_cameras_homography, I believe that the output will actually give you inverse focal length, so just do 1 / the focal length that it gives you.

Our code assumes all cameras are NDC Pytorch3D cameras.