bmild / nerf

Code release for NeRF (Neural Radiance Fields)
http://tancik.com/nerf
MIT License
9.58k stars 1.34k forks source link

Conversion of NeRF coordinate system back to original world coordinates #129

Open waps101 opened 2 years ago

waps101 commented 2 years ago

Prior to training, several coordinate system transformations are applied. This means that a mesh extracted from the NeRF is no longer in the world coordinates of the original camera system. I would like to be able to undo this series of transformations so that I can put the mesh back into original world coordinates. I think I know how to do this but it doesn't work...

Specifically, I am using the load LLFF code, factor=1, spherify set to true and recenter is set to false. This means there is

  1. A scaling applied here: https://github.com/bmild/nerf/blob/master/load_llff.py#L258
  2. A rotation and translation applied here: https://github.com/bmild/nerf/blob/master/load_llff.py#L208
  3. A scaling applied here: https://github.com/bmild/nerf/blob/master/load_llff.py#L213

Then obviously there is a scale and translate applied by the conversion to voxel coordinates in marching cubes. Undoing all of this almost works - scale and orientation looks right but there's always a significant difference in translation.

Has anyone managed to undo this series of transformations? Or is there an easier way of directly calculating the transformation to undo them?