refine_hloc triangulator not refining poses

[darfire]

Hi,

I am trying to refine a model from known poses and I'm having issues making it work.

First my data set. I have:

• almost 800 images, about 3-4 sequences
• poses for all of the images
• camera intrinsics

The poses and camera intrinsics both need some refining. I am trying to use this project to achieve that. My reference_model is in a colmap text format (with points3D.txt empty).

I run the following sequence of commands:

$ python -m hloc.extract_features --image_dir images/ --feature_path features.h5 --export_dir export
$ colmap model_converter --input_path ref_model/ --output_path ref_model_bin --output_type BIN
$ python -m hloc.pairs_from_poses --output pairs.h5 --model ref_model_bin/ --num_matched 50
$ python -m hloc.match_features --pairs pairs.h5 --export_dir export/
$ python -m pixsfm.refine_hloc triangulator --sfm_dir sfm_test --image_dir images/ --pairs_path pairs.h5 --features_path export/feats-superpoint-n4096-r1024.h5 --matches_path export/feats-superpoint-n4096-r1024_matches-superglue_pairs.h5 --reference_sfm_model ref_model_bin/ --config low_memory

At the end I get 2 models, one in sfm_test and one in sfm_test/hloc. I converted both to TXT and inspected cameras.txt and images.txt. Both the camera intrinsics and a selection of poses seem identical up to 12 decimals to the reference.

The reference poses themselves are pretty close but not close enough for what I am trying to achieve on this dataset (nerf).

Any suggestion on what I may be doing wrong here would be hugely appreciated.

[Phil26AT]

Hi, triangulation only refines 3D points. If you want to refine 3D-points + poses + intrinsics, you can just use python -m pixsfm.refine_colmap bahttps://github.com/cvg/pixel-perfect-sfm/blob/17f9df35d02cc06bcdb8b072a739cd447b38e930/pixsfm/refine_colmap.py#L82-L95 after you built your 3D model from hloc triangulation.

If you use the low_memory config, be sure to allow BA to refine poses and intrinsics by setting these parameters to true.