Hii, I have a question about the comments in sparse_scene_optimizer.py. As this part seems not to be in the paper (please correct me if I'm wrong ^^) I want to first start with a clarification question:
Coarse step: _makepts3d projects 2D pixel coord to 3D world coord by $\text{C2W} z K^{-1} * (u,v)$. Then, we calculate the error for aligning the the same 3D points derived from the pair.
Fine step:reproj2d projects 3D world coord to 2D pixel coord by $(K \text{W2C} (X,Y,Z))\ / \ z$. Then, we calculate the reprojection error, namely an aligned 3D point should be projected to the corresponding 2D pixels in both views.
In this line, the comment says don't optimize 3d points. Since the following code disables the gradient for pps, log_focals, core_depth, and these params are related to calculate the position of 3D points, does this comment imply coarse step only optimizes camera poses?
Then, in the fine step, all parameters receive gradients. Does this mean the fine step optimizes camera poses and 3D points position simultaneously?
Hii, I have a question about the comments in sparse_scene_optimizer.py. As this part seems not to be in the paper (please correct me if I'm wrong ^^) I want to first start with a clarification question:
Is my understanding correct?
Now, given my understanding above,
https://github.com/naver/mast3r/blob/b587bb36c11351e9303517d3940a0cd3d750cae6/mast3r/cloud_opt/sparse_ga.py#L430
In this line, the comment says don't optimize 3d points. Since the following code disables the gradient for pps, log_focals, core_depth, and these params are related to calculate the position of 3D points, does this comment imply coarse step only optimizes camera poses?
Then, in the fine step, all parameters receive gradients. Does this mean the fine step optimizes camera poses and 3D points position simultaneously?
I appreciate very much your help. Thanks!