Error in evaluating the ResidualBlock, calculated nan residuals

[cre185]

Hi, I was running my script for sparse reconstruction using glomap (and colmap). my script as below:

source_path=$1
source_path=$(echo $source_path | sed 's:/*$::')
colmap feature_extractor --image_path "$source_path/images/" --database_path "$source_path/database.db"
colmap exhaustive_matcher --database_path "$source_path/database.db"
glomap mapper --database_path "$source_path/database.db" --image_path "$source_path/images/" --output_path "$source_path/sparse/"

I think it's a loyal implementation of the 'From images' part in README, just need to provide a folder with an 'images' folder inside it in COLMAP manner. But I encountered the following error while running on my own collection of images, it happened when the glomap mapper line was executed. The error looks like:

-------------------------------------
Running view graph calibration ...
-------------------------------------
I0731 11:04:33.327239  8420 view_graph_calibration.cc:14] Start ViewGraphCalibrator
W0731 11:04:33.328502  8420 residual_block.cc:127]

Error in evaluating the ResidualBlock.

There are two possible reasons. Either the CostFunction did not evaluate and fill all
residual and jacobians that were requested or there was a non-finite value (nan/infinite)
generated during the or jacobian computation.

Residual Block size: 2 parameter blocks x 2 residuals

For each parameter block, the value of the parameters are printed in the first column
and the value of the jacobian under the corresponding residual. If a ParameterBlock was
held constant then the corresponding jacobian is printed as 'Not Computed'. If an entry
of the Jacobian/residual array was requested but was not written to by user code, it is
indicated by 'Uninitialized'. This is an error. Residuals or Jacobian values evaluating
to Inf or NaN is also an error.

Residuals:             -nan         -nan

Parameter Block 0, size: 1

     3857.14 | Not Computed  Not Computed

Parameter Block 1, size: 1

     693.486 | Not Computed  Not Computed

I0731 11:04:33.328533  8420 view_graph_calibration.cc:155] 0 cameras are rejected in view graph calibration
W0731 11:04:33.328562  8420 residual_block.cc:127]

I checked the output of previous steps, nothing seems gone wrong in feature extraction, and feature matching said I0731 11:04:33.316403 8420 colmap_converter.cc:301] Pairs read done. 99 / 220 are invalid, which I consider as a normal one. Also preprocess output I0731 11:04:33.327173 8420 view_graph_manipulation.cc:294] Decompose relative pose done. 37 pairs are pure rotation. I don't know if I've made any mistakes, or the error is due to anything else?

[jot-jt]

I initially had this error when I built GLOMAP. I fixed it by updating GLOMAP library dependencies, updating gcc to 12.4.0, and rebuilding.

Edit: my installation still has errors on internet photos, see below

[cre185]

New discovery: I used another dataset made up of images extracted from a video (thus very consistent and clear) without any modification in my environment, and it worked and generated normally. So it might actually be a dataset problem, the images I used formerly is a collection of internet photos so they may be especially noisy, the problem is I still want to use GLOMAP to handle the photos. Should there be a better way to achieve this? Like alter some parameters? I just don't know how to avoid the error. @jot-jt , thanks for your idea, I may try it later to find out, and may I ask you have you met the situation I mentioned above?

[jot-jt]

@cre185 Now that you mention it, I just tried to run my current GLOMAP installation on a scene of internet photos. I also get nan residuals, so perhaps it is internet photos and not the installation. My current GLOMAP installation works on the example gerrard-hall scene in the README, which has photos taken by only one camera.

(Specifically, the internet photo scene I used was Gendarmenmarkt from the 1DSfM dataset.)

[lpanaf]

Hi, thanks for your feedback. Is it possible for you (@jot-jt @cre185) to share your database? Either here, or you can send me an email if you don't want to make it public.

[cre185]

@lpanaf I'm just trying to apply GLOMAP to Megascenes: https://github.com/MegaScenes/dataset, which is really an 'in-the-wild' one. I'm not very surprised at the result, because the dataset is very challenging for those tasks. Anyway, I used to input these images into COLMAP, and some of the cameras can be reconstructed successfully while others can't. I'm just wondering if there could be a better way for GLOMAP to handle these noisy datasets, rather than throwing out an error. Maybe it's just not suitable for global reconstruction methods. btw, just noticed that @jot-jt is one of the authors of Megascenes, what a coincidence...

[jot-jt]

@lpanaf Thanks for looking into this! I uploaded the database file here: Google Drive link

The images can be downloaded here: http://landmark.cs.cornell.edu/projects/1dsfm/images.Gendarmenmarkt.tar

[Dnotitiajhcha]

I am trying to run the glomap mapper command using a database.db created through Deep Image Matching , but I am encountering the same error. However, when I create the database.db using COLMAP as described on this GitHub page and then use it with the glomap mapper command, no errors occur.

[ahojnnes]

Thanks for sharing your database to reproduce the issue. This should now be fixed in the latest commit in the main branch.