Guidance on improving SFM reconstruction

[fsblu]

Hello, Thank you for this project. I have a question regarding working with my own dataset for reconstruction. SFM construction steps are quite clear, and seem to work. However, when I use my own data (extracted images of a video taken while going around a block), reconstruction step outputs half of the expected model (maybe only front and sides of the block. Back of the block is not part of the model). Here are my questions:

1. Is generated model 0 combination of all submodels?
2. As someone with experience, do you have any suggestion on how to improve reconstruction? I want to obtain entire block model, not just half of it.
   • I have tried working with different number of images. The result is same.
   • Image retrieval step seems to work. I have checked it for random images.

Thank you!

[sarlinpe]

1. No, the submodels are disjoint and we generally select the submodel with the most images.
2. Two submodels can emerge for diverse reasons:
   • at some point the motion is too large and image matching does not provide any constraint
   • lots of incorrect matches, due to symmetries or repeated patterns, make the reconstruction globally consistent.

Since you are dealing with sequences, you can be more careful with the selection of the image pairs using both:

• sequential pairs between each image and its K previous ones. We don't have a script for that, you're welcome to PR one with a new script pairs_from_sequence.
• loop closure pairs with image retrieval, triggered every N frame (depending on the framerate + motion magnitude).

[soskek]

If the camera parameters (focus and distortion) of the video are constant over the frames, I guess that it could improve the results to use the single camera mode. By default, reconstruction assumes that all the images can have different camera parameters.

For example, I changed

model = reconstruction.main(sfm_dir, images, sfm_pairs, features, matches, image_list=references)

to

model = reconstruction.main(sfm_dir, images, sfm_pairs, features, matches, image_list=references, camera_mode=pycolmap.CameraMode.SINGLE)

[sarlinpe]

I assumed that the camera were already shared, thanks @soskek for highlighting this. In hloc v1.3, we have a new parameter camera_mode which has the default value AUTO. This mode shares the same camera among images that have identical EXIF tags:

// Note: When using AUTO mode a camera model will be uniquely identified by the // following 5 parameters from EXIF tags: // 1. Camera Make // 2. Camera Model // 3. Focal Length // 4. Image Width // 5. Image Height

In hloc v1.3 the default value of camera_mode should be fine if all images were captured by the same camera on the same date. To be on the safe side you should check this after the reconstruction.