Scaling factor of poses

[basit-7]

COLMAP uses some factor to scale down poses and intrinsics. Where can I find this scaling factor? Is it fixed or does it depend on the images?

[tsattler]

The scaling factor is random and comes from the fact that it is impossible to determine the size of the scene from 2D measurements alone if you do not know any distances in 3D (think of it this way: you could have images showing a closeup of a very small scene or images showing a very large scene from far away; without knowing which of the cases you are in, these are impossible to distinguish). The scaling factor depends on which image pair is used to start the reconstruction process. To find the scaling factor, you need to know at least one distance between points in a scene (either the distance between two 3D points, the distance between a camera and a 3D point, or between two cameras). You can then compute the same distance in the model and the ratio between the former and the latter gives you the scaling factor.

[basit-7]

@tsattler thanks for explaining in great detail. Just one last thing, does it use the same factor to scale intrinsics as well as extrinsic?

If yes, can I use the Ground truth focal length and the one predicted by COLMAP to find the scaling factor?

[tsattler]

I am not sure what you mean by the scaling factor for the intrinsics. While it is impossible to recover the size of the scene from images, it is possible (up to noise in the process and a some degenerate configurations aside) to fully recover the intrinsic calibration.

To get the scaling factor that converts units in 3D into physically meaningful units, e.g., meters, you need to know distances in 3D.

[aarrushi]

For the poses, is there a common scaling factor across X,Y,Z directions? Or does it scale along each axes differently?

[tsattler]

There is a single scaling factor.

[Kirang96]

@tsattler, If I have the distance from camera to the object, how do I compute the scaling factor from it? Can I just divide the camera to object distance with the same distance estimated by colmap to get the scaling factor?

Will the scaling factor be same for all the images captured for a single model? Or does it vary for each image?

[tsattler]

Can I just divide the camera to object distance with the same distance estimated by colmap to get the scaling factor?

Yes.

Will the scaling factor be same for all the images captured for a single model? Or does it vary for each image?

In theory it should be the same for all images. In practice, there will be noise and errors in the camera poses and the 3D points, so the scaling factor will vary (slightly).

[Kirang96]

Alright. If I'm assuming I have the real distance from the camera to the object, all I need is the distance from camera to the object estimated in colmap to get the scaling factor. I understand I can get the distance between two points in colmap by just calculating the Euclidean distance between them. How do I calculate the Euclidean distance from one of the cameras to the object in colmap?

[tsattler]

By computing the distance between the position of the point and the position of the camera in the scene.

[Kirang96]

Oh. alright. So tx,ty,tz in the database gives the position of the camera. Got it. Thanks a lot @tsattler!

[tsattler]

tx, ty, tz does not give you the position of the camera (see https://colmap.github.io/format.html#images-txt). Please note that the pose parameters in the database are not updated after reconstruction. You will need to use the poses in the reconstruction that you save.

[Kirang96]

Oh sorry, my bad. What I meant was the tx,ty,tz we get in the image information dialogue box when we click on one of the cameras in gui. It is the same value as in images.txt file.

[Kirang96]

Can I just divide the camera to object distance with the same distance estimated by colmap to get the scaling factor?

Yes.

@tsattler will this method work if I'm using the distance between two cameras instead of distance from camera to object. eg. I have the real distance between two images, then I get the distance between those images estimated by colmap, then divide the real distance by colmap estimate to get the scaling factor?

[tsattler]

Yes, this will work.

[Kirang96]

Yes, this will work.

@tsattler , I've tried doing this, but somehow the scaling factor is not correct. I have arcore priors which are pretty accurate. I verified this by calculating the distance between two known points using arcore. I created a reconstruction using colmap but did not input any priors (I have the pose but did not use it). Then calculated the distance between two cameras as estimated by colmap.

colmap_image_1 = 3.97439, -2.64522, 2.02285
colmap_image_2 = -3.94725, -1.94046, 2.38913
Euclidean distance = 7.9613

Then I used the poses I had and got the distance between these two images:

arcore_image_1 = -1.1549344, -0.80167365, 0.96329045
arcore_image_2 = -0.44475895, -0.8635335, 0.9316467
arcore euclidean distance = 0.7135 (meters)

Then I calculated the scaling factor as 0.7135/7.9613 = 0.0896

With this scaling factor I tried to scale the distance between two points in the colmap where the real distance is known, but the result is not correct. I'm getting 35.4cm in colmap after scaling while the real distance is 28cm.

Is there something else I should do before doing this calculation?

[tsattler]

If you just use a single pair, your scaling factor will be affected by noise in the individual pose estimates (both arcore and Colmap produce noisy poses). It would be better to use all pairs to estimate the scale. You can also use the model_aligner to find a transformation that aligns the Colmap poses to the arcore poses.

[Kirang96]

I tried getting scaling factor from multiple pairs and taking the average. But still the measurements are not correct. I verified the arcore translations by moving the device from one point to another as well, so the translations are good.

[tsattler]

Have you tried the model_aligner?

[Kirang96]

Have you tried the model_aligner?

For calculating the scaling factor from multiple pairs, I reconstructed the model in GUI and saved it as text files. Then used it for calculating the scale factor with original poses. But the model_aligner output is as bin files and I'm unable to read it in order to get the scaling factor. Is it possible to save the model_aligner output as text files?

[tsattler]

You can convert binary files to different formats, including text files, using the model_converter.

[Kirang96]

@tsattler, model_aligner works! Now the model is up to scale. Thanks for helping me out.

[kevinchan04]

@tsattler, model_aligner works! Now the model is up to scale. Thanks for helping me out.

Hi, may I ask how to get the scale from the result of model_aligner?

[Kirang96]

@tsattler, model_aligner works! Now the model is up to scale. Thanks for helping me out.

Hi, may I ask how to get the scale from the result of model_aligner?

model_aligner only aligns the sparse model, it does not output the scale. But we can output the transformation matrix using transform_path argument. I'm not sure how to get the scale from that though.

[kevinchan04]

@Kirang96 Thanks for your reply! Is it the transform matrix can scale up the model from colmap to real world? In your case, the coordinate of arcore.

[Kirang96]

@Kirang96 Thanks for your reply! Is it the transform matrix can scale up the model from colmap to real world? In your case, the coordinate of arcore.

Yes. In my knowledge, model_aligner first computes this transformation matrix using the arcore poses and aligns the sparse point cloud to real world coordinates. But if you want to get the scaling factor I'm not sure how to get it. I think the scale is multiplied with the 3x3 rotation matrix.

[AnFeiFei]

@tsattler, model_aligner works! Now the model is up to scale. Thanks for helping me out.

Can I really restore the scaling ratio of the real world? I am trying to use model-aligner to restore the true scale. But it feels like it hasn't been successful. My experimental process is as follows:

1. Place the object at position A, and after generating a sparse point cloud, use model-aligner for coordinate transformation to obtain point cloud 1.

2. Place the object at position A and translate it to position B, and also use model-aligner for coordinate transformation to obtain point cloud 2.

In the end, the proportions of objects in point cloud 1 and point cloud 2 are not consistent, and they are not in the same plane.

[tsattler]

The model_aligner scales the Colmap model based on the reference data that you provide. This only recovers the scale in the real world if the reference data is in real world coordinates.

[oUp2Uo]

@tsattler, model_aligner works! Now the model is up to scale. Thanks for helping me out.

Hi, could you please tell more detail about how to use model_aligner? I have tried many parameters, but some only could move the origin point to the 1st image. The scale of the axes didn't change in most cases(using enu-plane-unscaled changed).

[nectorv]

what is the heuristic used to scale the scene ?

[lemonci]

Hello, if I use Set A with n images for reconstruction, and then use Set B containing all elements in Set A + a new image for reconstruction. Is there a way to estimate the camera pose of the new image in Set A's coordinate system based on the n comman images? Thanks.