Translation (t_z) and focal length in fit-model-ceres

[Sun1718]

Hi, I run fit-model-ceres with default image and pts file as input (image_0010.png). It works good when use_perspective flag is false. But I have problem when use_perspective flag is true. All parameters calculated correctly instead of t_z. It return a wrong t_z value (for example -1842.23 for the default image or -7543.56 or -3432.78 for other images). Do you know what is problem or where am I wrong? So thanks.

[patrikhuber]

Hi, Estimating perspective (focal length / t_z) from a single image is an ill-posed problem with infinitely many solutions. You could use some prior knowledge, e.g. if you have the focal length from the camera. Many SfM methods use this. Some 3DMM fitting methods also just set the focal length to more or less constant. Video makes it less ill-posed too. You can read some further literature about this, for example William A. P. Smith, The Perspective Face Shape Ambiguity. PS: How do you know the values you're getting are "wrong"? Keep in mind I think the camera looks down the negative z axis, that would be why they're negative.

[Sun1718]

Thanks for your reply. I've tested it again and I found it was my mistake in projection matrix construction. I thought a big value for t_z was unusual. But it workes :-)

The main problem is it seems same as none perspective(ortho) camera. I think it is because of its big z value which destroying perspective. I think it will be correct if I scale down my model first but I can't find correct scale value. What is your idea about it?

I have a video so I can use multiple images for estimating perspective camera. Is your multi_image_fit_devel for this purpose? Can I use multiple images for estimating fov and t_z using first frames then use those values as constant for the next frames?

Thanks for your time.

[patrikhuber]

Great :-)

The perspective camera model doesn't have a scale parameter, the size of the face is determined by f and t_z. You can read more about the camera models in e.g. H&Z.

The multi_image_fit_devel branch (and once it's merged) is for fast, linear, SOP multi-frame fitting. Not for perspective and not using Ceres. You can easily extend the cost function of fit-model-ceres to go over multiple frames (with a for loop).

[Thanh-Binh]

I did not find the branch multi images fit develop. Could you pls show me where it is? Thanks

[Sun1718]

Thanks Patrik. I appreciate your help. I'll work on it further.

To Thanh-Binh, You can find it under this link: https://github.com/patrikhuber/eos/branches

[patrikhuber]

I'll close this ticket as it's not really an issue with eos. And the linear multi-frame fitting is now merged into master :-)