search

facebookresearch / localrf

An algorithm for reconstructing the radiance field of a large-scale scene from a single casually captured video.
MIT License
956 stars 62 forks source link

Render from file poses - out of RF bounds #39

Open NikitaPovelkin opened 10 months ago

NikitaPovelkin commented 10 months ago

Hi,

great paper and method!

I am having an issue with the "render from file" Method, where given poses have to be used for the render of an image. Even if i use the same poses for the training and then for the render from file, the results of a render are awful, since the training poses dont match the render poses (although both pose files are exactly the same). I've managed to find out that during the training, from poses - relative poses (rel_poses) are generated and then used for appending scenes etc. After doing similar operation with the poses for the rendering, the result is not out of bounds anymore, but is still doesn't off by a bit. After comparing the rel_poses to the poses, which are written into transforms.json at the end of the training, i found out, there are differences between the two.

So, my question ist, how to train with preprocessed poses and render from file, so that the poses match?

Best regards, Nikita

ameuleman commented 9 months ago

Hi, Thanks for pointing out this problem, I will take a look.

ameuleman commented 8 months ago

Hi, A scene optimized with --with_preprocessed_poses 1 --lr_R_init 0 --lr_t_init 0 can now be rendered with --render_from_file properly.