Image positions numerical errors with hierachical mapper

[Tezirg-Wrld3D]

From 12 videos sampled every 10 frames, is created a dataset of ~6k images. The features are extracted using default parameters, then a sequential matching pass is performed, followed by features extraction and vocab_tree matching of a set a 20 images took at the spatial start/end of the videos. Last, 3 iterations of the transitive matcher is used to join the frames. According to the log all the frames are linked in a single graph after this process.

Finally the hierarchical mapper is used to create a sparse model: Nothing suspicious appears in the logs, as the images are registered and the bundle adjustments terminate on convergence.

Colmap outputs 3 directories, the first two models having around 10 images and the last model images.bin referencing the rest of the images. The problem is that the images values for qw,qx,qy,qz,tx,ty,tz are nan or -nan.

Is there some parameter or error in the process that could lead to this result ?

[ahojnnes]

That sounds strange. Are you referring to the database or the images.bin when you talk about the nan’s?

[Tezirg-Wrld3D]

I'm referring to the images.bin / images.txt. Alternatively, if reconstructed independently, some videos reconstructions are erroneous because of very sharp reflections in computer screens or because the camera is just rotating in place. I think that might cause an issue when trying to bundle adjust with frames from valid videos. I will try to start again from scratch excluding these videos to see if the problem persists.

EDIT: Removing these doesnt solve the issue, most of the camera positions/rotations are NaN

[isolin]

I can confirm that I encountered the same issue in my dataset. I had to filter out all the NaNs using an own script to be able to display the sparse point cloud in CloudCompare or Potree. Yet unsure how the errors will affect the dense reconstruction I plan.

I produced the dataset using the hierarchical_mapper. During feature extraction and matching I enabled

SiftExtraction.estimate_affine_shape=true 
SiftExtraction.domain_size_pooling=true
SiftMatching.guided_matching=true

[wjohnson191]

I'm experiencing this same issue after running hierarchical_mapper on COLMAP 3.6 (Commit c423ca9 on 2019-03-14 with CUDA).

Out of 2732 images, all but 10 of them look like this in images.txt:

$ grep -E "^[0-9]+ nan .+\.jpg" images.txt | head -n 5
562 nan nan nan nan nan nan nan 1 1554223564603_0000562.jpg
2650 nan nan nan nan nan nan nan 1 1554223564603_0002650.jpg
599 nan nan nan nan nan nan nan 1 1554223564603_0000599.jpg
2648 nan nan nan nan nan nan nan 1 1554223564603_0002648.jpg
597 nan nan nan nan nan nan nan 1 1554223564603_0000597.jpg

$ grep -E "^[0-9]+ nan .+\.jpg" images.txt | wc -l
    2722

Here are the 10 images that are non-nan for QW, QX, QY, QZ, TX, TY, TZ:

$ grep -E "^[0-9]+ [^n].+\.jpg" images.txt
555 0.933286 0.0150038 0.351378 0.0727073 -8.81786 -0.118947 0.983858 1 1554223564603_0000555.jpg
554 0.953398 0.00916773 0.293368 0.069876 -6.3834 -0.253495 0.34728 1 1554223564603_0000554.jpg
553 0.97005 -0.000988921 0.235888 0.0579527 -4.11001 -0.455206 0.0225472 1 1554223564603_0000553.jpg
552 0.983984 -0.00197057 0.172744 0.0439404 -1.81038 -0.379097 -0.102927 1 1554223564603_0000552.jpg
551 0.993128 -0.00346071 0.112662 0.0314827 0.169794 -0.437923 0.0442497 1 1554223564603_0000551.jpg
550 0.996743 0.000626037 0.0778674 0.0209767 1.07068 -0.352518 0.380073 1 1554223564603_0000550.jpg
549 0.998525 0.0033364 0.052583 0.0131186 1.73337 -0.289006 0.707201 1 1554223564603_0000549.jpg
548 0.999509 0.00510733 0.0305929 0.00453363 2.30004 -0.270102 1.06753 1 1554223564603_0000548.jpg
547 0.9999 0.00692361 0.0116203 -0.00419388 2.79018 -0.295799 1.41828 1 1554223564603_0000547.jpg
546 0.999806 0.0106036 -0.00873535 -0.0140836 3.05436 -0.309245 1.74045 1 1554223564603_0000546.jpg

Note that matching POINTS2D -> POINTS3D still seems to have been successful for the images that end up all-nan:

2650 nan nan nan nan nan nan nan 1 1554223564603_0002650.jpg
1886.47 27.6003 -1 1768.65 31.5371 779789 1737.16 55.9387 779787 1755.6 61.5894 779785 1755.6 61.5894 779783 1745.72 65.2764 779781 1745.72 65.2764 779779 1706.52 81.2195 780631 1743.47 86.95 779777 1743.47 86.95 804883 3013.01 88.3132 -1 1733.41 92.0747 778975 1733.41 92.0747 778977 ...

However, the corresponding POINTS3D have nan's as well for X, Y, Z and ERROR in these cases:

$ grep -E "^779789 " points3D.txt
779789 nan nan nan 211 220 249 nan 2619 10 2650 1 2618 9 2648 2 2612 6 2629 8

I ran hierarchical_mapper with default settings, as such:

colmap hierarchical_mapper \
    --database_path database.db \
    --image_path images \
    --output_path sparse

Like @Tezirg-Wrld3D, these frames are from a video, downsampled to 6fps. The camera was moving continuously.

Any ideas about how I could debug/avoid this? Thanks in advance.

[ghasemikasra39]

In my case, all the images have nan values:

I am using OPENCV_FISHEYE for my 360 degree images for this dataset. I tried giving the correct intrinsics and distortaiton parameters, but still no success. (I also tried giving it nothing and let colmap figure out everything, but no success again) I can also see that the camera extrinsics computed by colmap, are all nan values in the database. (Is it expected?) Does Colmap has support for my case?

Dense reconstruction	Database