search

facebookresearch / localrf

An algorithm for reconstructing the radiance field of a large-scale scene from a single casually captured video.
MIT License
956 stars 62 forks source link

Two ideas and ask if they are feasible #37

Closed dustier closed 10 months ago

dustier commented 10 months ago

Hi, great work. Have two ideas and would like to ask if they are feasible:

  1. Is it possible to input the pose of each image instead of relying on the algorithm to predict it? Sometimes there may be better algorithms or other sensors to assist in predicting the pose of each input image, such as feature extraction and matching based on deep learning, lidar-aided pose calculation, etc.
  2. Supposing the position at timestamp $t_0$ and $t_n$ are near, is the radiance field optimizing at $t_n$ the same as the radiance field at $t_0$? If not, is it feasible to optimize the same radiance field?

I know the current code may not support the above features. Just want to ask if the above two ideas conflict with the existing algorithm framework and whether they are theoretically feasible?

ameuleman commented 10 months ago

Hi,

  1. Yes, with_preprocessed_poses 1 --lr_R_init 0 --lr_t_init 0 disables pose optimization and loads camera parameters from ${SCENE_DIR}/transforms.json.
    1. This should be feasible. However, it will likely require some loop closure detection as some pose drift may occur with the current local pose estimation.