sqlite3.IntegrityError: UNIQUE constraint failed: images.name When localization on RobotCar_Seasons-v2

[AddASecond]

What did I do:

For RobotCar_Seasons-v2 dataset, At first I use all mapping images to create the map, this action got the "sqlite3.IntegrityError: UNIQUE constraint failed: images.name" error during the mapping step after that I try to use only the 01 folder to create the map, this action got the "sqlite3.IntegrityError: UNIQUE constraint failed: images.name" error during the localization step

Question:

I wonder why this error happens? There should be no overlap between query and mapping images.name

shell script :

'# 7) localization pipeline LOCAL=r2d2_WASF_N8_big GLOBAL=Resnet101-AP-GeM-LM18 kapture_pipeline_localize.py -v info -f \ --benchmark-style RobotCar_Seasons \ -i ${WORKING_DIR}/${DATASET}/01/mapping \ --query ${WORKING_DIR}/${DATASET}/query \ -kpt ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/keypoints \ -desc ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/descriptors \ -gfeat ${WORKING_DIR}/${DATASET}/global_features/${GLOBAL}/global_features \ -matches ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/NN_no_gv/matches/01 \ -matches-gv ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/NN_colmap_gv/matches/01 \ --colmap-map ${WORKING_DIR}/${DATASET}/colmap-sfm/${LOCAL}/${GLOBAL}/01 \ -o ${WORKING_DIR}/${DATASET}/colmap-localize/${LOCAL}/${GLOBAL}/ \ --topk ${TOPK} \ --config 2'

error info:

'/mine-run_robotcar-v2.sh INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_mapping sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/01/mapping/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_query sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/query/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/pairs_localization_20.txt INFO ::compute_image_pairs: compute_image_pairs. loading mapping: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_mapping INFO ::compute_image_pairs: computing pairs with Resnet101-AP-GeM-LM18... INFO ::compute_image_pairs: compute_image_pairs. loading query: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_query INFO ::compute_image_pairs: saving to file ... INFO ::compute_image_pairs: all done INFO ::merge: Loading /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_mapping INFO ::merge: Loading /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_query INFO ::merge: Writing merged kapture data... INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/map_plus_query/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::compute_matches: compute_matches. loading input: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query INFO ::compute_matches: compute_matches. entering main loop... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 124320/124320 [00:00<00:00, 321940.37it/s] INFO ::compute_matches: all done INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query_gv sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/map_plus_query/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::run_colmap_gv: run_colmap_gv... INFO ::run_colmap_gv: remove rigs notation. colmap_dbi_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query_gv/colmap.db INFO ::colmap: registering 135 sensors (cameras) in database... INFO ::colmap: registering 12996 images in database... Traceback (most recent call last): File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 203, in run_colmap_gv_command_line() File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 196, in run_colmap_gv_command_line run_colmap_gv(args.input, args.output, args.colmap_binary, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 47, in run_colmap_gv run_colmap_gv_from_loaded_data(kapture_none_matches, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 111, in run_colmap_gv_from_loaded_data database_extra.kapture_to_colmap(kapture_data_to_export, kapture_none_matches_dirpath, File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database_extra.py", line 641, in kapture_to_colmap colmap_image_ids = add_images_to_database( File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database_extra.py", line 493, in add_images_to_database colmap_image_ids = add_images_from_list_in_colmap_format(database, images_in_colmap_format) File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database_extra.py", line 416, in add_images_from_list_in_colmap_format colmap_image_ids[name] = database.add_image( File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database.py", line 172, in add_image cursor = self.execute( sqlite3.IntegrityError: UNIQUE constraint failed: images.name Traceback (most recent call last): File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_localize.py", line 394, in localize_pipeline_command_line() File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_localize.py", line 363, in localize_pipeline_command_line localize_pipeline(args.kapture_map, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_localize.py", line 187, in localize_pipeline run_python_command(local_run_colmap_gv_path, run_colmap_gv_args, python_binary) File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture_localization/utils/subprocess.py", line 67, in run_python_command raise ValueError('\nSubprocess Error (Return code:' f' {python_process.returncode} )') ValueError: Subprocess Error (Return code: 1 )'

[yocabon]

Hi, there isn't any duplicate in RobotCar_Seasons-v2 image names could you tell me more about how you obtained your RobotCar_Seasons-v2/query data ?

it crashes at the geometric verification step Here I read INFO ::colmap: registering 135 sensors (cameras) in database... which is not coherent with the dataset you're using. RobotCar_Seasons-v2 have query data for slices 04 05 06 23 24 25 26 27 28 29 30 31 32 33 34 35 36 44 45 47 48 49, that's 22. If you built your map with 01/mapping, and merged without keeping the sensor ids, you should have at this step 3 + 22x3=75 sensors I believe.

[AddASecond]

Hi, yocabon

The RobotCar_Seasons-v2/query data is simply obtained by merge all query data only in 49 folders by using: kapture_merge.py -v debug \ -i $query $query \ -o query \ --image_transfer link_relative

here is my full shell script for RobotCar_Seasons-v2 that cause the error info above, during the mapping and localization step: https://gist.github.com/AddASecond/58f96176b6235ff858bad019ff1dd3d6

where I adapt your script here https://github.com/naver/kapture-localization/blob/main/doc/benchmark.adoc#robotcar-seasons to Aachen example shell script here: https://github.com/naver/kapture-localization/blob/main/pipeline/examples/run_aachen-v11.sh here

would you give some advice? thank you very much

I paster the error info again as follows(which is exactly the output of the gist script above):

INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_mapping sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/mapping/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_query sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/04/query/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/pairs_localization_20.txt INFO ::compute_image_pairs: compute_image_pairs. loading mapping: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_mapping INFO ::compute_image_pairs: computing pairs with Resnet101-AP-GeM-LM18... INFO ::compute_image_pairs: compute_image_pairs. loading query: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_query INFO ::compute_image_pairs: saving to file ... INFO ::compute_image_pairs: all done INFO ::merge: Loading /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_mapping INFO ::merge: Loading /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_query INFO ::merge: Writing merged kapture data... INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/map_plus_query/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::compute_matches: compute_matches. loading input: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query INFO ::compute_matches: compute_matches. entering main loop... 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10080/10080 [00:00<00:00, 332310.88it/s] INFO ::compute_matches: all done INFO ::kapture: deleting already existing /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query_gv sensors in path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/map_plus_query/sensors keypoints_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/local_features/r2d2_WASF_N8_big/keypoints INFO ::run_colmap_gv: run_colmap_gv... INFO ::run_colmap_gv: remove rigs notation. colmap_dbi_path: /sdb3/myfolder/00projects/sfm/kapture-localization/pipeline/examples/RobotCar_Seasons-v2/colmap-localize/r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18/kapture_inputs/proxy_map_plus_query_gv/colmap.db INFO ::colmap: registering 297 sensors (cameras) in database... INFO ::colmap: registering 53664 images in database... Traceback (most recent call last): File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 203, in run_colmap_gv_command_line() File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 196, in run_colmap_gv_command_line run_colmap_gv(args.input, args.output, args.colmap_binary, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 47, in run_colmap_gv run_colmap_gv_from_loaded_data(kapture_none_matches, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_run_colmap_gv.py", line 111, in run_colmap_gv_from_loaded_data database_extra.kapture_to_colmap(kapture_data_to_export, kapture_none_matches_dirpath, File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database_extra.py", line 641, in kapture_to_colmap colmap_image_ids = add_images_to_database( File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database_extra.py", line 493, in add_images_to_database colmap_image_ids = add_images_from_list_in_colmap_format(database, images_in_colmap_format) File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database_extra.py", line 416, in add_images_from_list_in_colmap_format colmap_image_ids[name] = database.add_image( File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture/converter/colmap/database.py", line 172, in add_image cursor = self.execute( sqlite3.IntegrityError: UNIQUE constraint failed: images.name Traceback (most recent call last): File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_localize.py", line 394, in localize_pipeline_command_line() File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_localize.py", line 363, in localize_pipeline_command_line localize_pipeline(args.kapture_map, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_localize.py", line 187, in localize_pipeline run_python_command(local_run_colmap_gv_path, run_colmap_gv_args, python_binary) File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture_localization/utils/subprocess.py", line 67, in run_python_command raise ValueError('\nSubprocess Error (Return code:' f' {python_process.returncode} )') ValueError: Subprocess Error (Return code: 1 )

[AddASecond]

OK, problem solved, I misused the function kapture_merge.py

The WRONG script: kapture_merge.py -v debug \ -i $mapping $mapping \ -o query \ --image_transfer link_relative This will result in everything double in .txt, 49 2 * 3=294, Then when you query with 3 camera pose , it becomes: INFO ::colmap: registering 297 sensors (cameras) in database...

The RIGHT script: kapture_merge.py -v debug \ -i $mapping \ -o query \ --image_transfer link_relative

[AddASecond]

Hi, after I changed the merge script, for the mapping step it registered 49 *3 =147sensors, successfully run until facing the following Error in Bundle adjustment:

============================================================================== Retriangulation

=> Completed observations: 436030 => Merged observations: 74746393

============================================================================== Bundle adjustment

Traceback (most recent call last): File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_colmap_build_map.py", line 270, in colmap_build_map_command_line() File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_colmap_build_map.py", line 262, in colmap_build_map_command_line colmap_build_map(args.input, args.output, args.colmap_binary, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_colmap_build_map.py", line 62, in colmap_build_map colmap_build_map_from_loaded_data(kapture_data, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_colmap_build_map.py", line 171, in colmap_build_map_from_loaded_data colmap_lib.run_point_triangulator( File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture_localization/colmap/colmap_command.py", line 275, in run_point_triangulator run_colmap_command(colmap_binary_path, point_triangulator_args) File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture_localization/colmap/colmap_command.py", line 69, in run_colmap_command raise ValueError( ValueError: Subprocess Error (Return code: -9 ) Traceback (most recent call last): File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_mapping.py", line 256, in mapping_pipeline_command_line() File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_mapping.py", line 230, in mapping_pipeline_command_line mapping_pipeline(args.kapture_map, File "/sdb3/myfolder/softwares/anaconda3_gy/bin/kapture_pipeline_mapping.py", line 144, in mapping_pipeline run_python_command(local_build_map_path, build_map_args, python_binary) File "/sdb3/myfolder/softwares/anaconda3_gy/lib/python3.8/site-packages/kapture_localization/utils/subprocess.py", line 67, in run_python_command raise ValueError('\nSubprocess Error (Return code:' f' {python_process.returncode} )') ValueError: Subprocess Error (Return code: 1 )

[yocabon]

Did you run out of memory ?

[AddASecond]

Did you run out of memory ?

Hi， thanks for you reply, I find it's a OOM through looking at the list of linux's 'top' command. at the mapping stage kapture uses about 26,000+ * top20 = 530,000 + matches on RobotCar-Seasons-V2

There are 2 questions: 1.How much memory will RobotCar-Seasons-V2 use? my computer has 128g memory, still not sufficient 2.Is it possible to change the code/deploy methods to reduce the memory usage while keeping the accuracy？ -I think only the relative images needs to be loaded into memory. I wonder the cause of this large memory, is it because all descriptors are loaded into memory? Notice that one image's descriptor is 9.8M(r2d2_WASF_N8_big/Resnet101-AP-GeM-LM18).

[yocabon]

1. I don't really know. I would usually process RobotCar-Seasons-V2 slice per slice, not the whole dataset at once; but I do have built models on the whole RobotCar-Seasons-V1 dataset, with about 1M matches, but it was either with 512GB or 1TB ram (I remember having some issues with memory at first when I was trying to build this).

2. It's crashing within colmap point_triangulator, not much I can do about that unfortunately. The best would be to process slice by slice; since you know they are not overlapping, or play with the number of keypoints / pairs (for the mapping, you won't have this problem with localization) you use.

Note that descriptors cannot be the problem, those are not exported to colmap (see here so it's all keypoints and matches).

[AddASecond]

Thanks a lot. working well with slice-per-slice method