search

rover-xingyu / L2G-NeRF

[CVPR 2023] L2G-NeRF: Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fields
https://rover-xingyu.github.io/L2G-NeRF/
MIT License
249 stars 6 forks source link

Question about local warp module #10

Closed zhywanna closed 7 months ago

zhywanna commented 9 months ago

Thanks for your great job! During reading your code, I have a problem can't figure out.

https://github.com/rover-xingyu/L2G-NeRF/blob/6fbac3261678cc8791a6834c559b26c04b7b8b7a/model/l2g_nerf.py#L252

Why don't you use camera_cords_grid_3D directly, but transfer to camera_cords_grid_2D before put in warp_mlp？ Is that necessary?

Thanks again! :)

rover-xingyu commented 9 months ago

Thanks for your interest! We use camera_cords_grid_2D for the sake of minimal coordinates representation, but theoretically speaking, using camera_cords_grid_3D directly would also work for you.

zhywanna commented 9 months ago

Thanks for your interest! We use camera_cords_grid_2D for the sake of minimal coordinates representation, but theoretically speaking, using camera_cords_grid_3D directly would also work for you.

Do you mean that using 2D camera coordinates are enough to refine local pose? Or the "local pose" is corresponding to 2D. It's still confusing to me.

And why do we only remove z-axis?

rover-xingyu commented 9 months ago

The camera_cords_grid_2D refers to pixel coordinates, which is enough for the network to identify different pixels and map them to local transformations.