yocabon
commented
3 years ago Hi, I'll answer the additional question since it's probably related to the low results you got. In kapture_image_retrieval_late_fusion.py, -i is the mapping data; not the map_plus_query data.
so in order to make that work, you would use
kapture_create_kapture_proxy.py with
-i ${WORKING_DIR}/${DATASET}/mapping -o ${WORKING_DIR}/${DATASET}/gharm/mapping -gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18/global_features ${WORKING_DIR}/${DATASET}/global_features/DELG/global_features ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi/global_features ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts/global_features(the resulting ${WORKING_DIR}/${DATASET}/gharm/mapping/reconstruction/global_features should have 4 folders)
and
kapture_create_kapture_proxy.py with
-i ${WORKING_DIR}/${DATASET}/query -o ${WORKING_DIR}/${DATASET}/gharm/query -gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18/global_features ${WORKING_DIR}/${DATASET}/global_features/DELG/global_features ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi/global_features ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts/global_featuresthen if you want gharm pairs for mapping (map <-> map -- which we would do unless we built a map with distance or many pairs), you would call
kapture_image_retrieval_late_fusion.py -v debug \
-i ${WORKING_DIR}/${DATASET}/gharm/mapping \
-o ${WORKING_DIR}/${DATASET}/pairs_fusion_mapping.txt \
--topk ${TOPK} generalized_harmonic_meanand for query -> map pairs
kapture_image_retrieval_late_fusion.py -v debug \
-i ${WORKING_DIR}/${DATASET}/gharm/mapping \
--query ${WORKING_DIR}/${DATASET}/gharm/query \
-o ${WORKING_DIR}/${DATASET}/pairs_fusion_query.txt \
--topk ${TOPK} generalized_harmonic_meannote that the -gfeat parameter is only here to help you select the global features types you want to use, in this case you would do -gfeat AP-GeM-LM18 DELG netvlad_vd16pitts for example if you didn't want to use netvlad.
I hope this helps.
AddASecond
humenbergerm
Question
During reproducing the late fusion step on the Aachen Day-Night v1.1 data set(with 4 global features that kapture provided, AP-GeM-LM18/ DELG/ densevlad_multi/ netvlad_vd16pitts/)，I found the accuracy at night is very low compared to the results in the paper.
with gharm Top20 config2, I got
I wonder whether it‘s an implementation problem or is there any other trick?
What did I do：
Step1: use full dataset to construct a single map according to https://github.com/naver/kapture-localization/blob/main/pipeline/examples/run_aachen-v11.sh
Step2: do late fusion using https://github.com/naver/kapture-localization/blob/823f85430c4739b398b5a1cf11ef7d942b0e917d/tools/kapture_image_retrieval_late_fusion.py Since there is no example of fusion script, I wrote one based on my understanding:
`# 0a) Define paths and params PYTHONBIN=python3.8 WORKING_DIR=${PWD} DATASETS_PATH=${WORKING_DIR}/datasets DATASET=Aachen-Day-Night-v1.1 mkdir -p ${DATASETS_PATH}
TOPK=20 # number of retrieved images for mapping and localization KPTS=20000 # number of local features to extract
-gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18/global_features ${WORKING_DIR}/${DATASET}/global_features/DELG/global_features ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi/global_features ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts/global_features \
-gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18 ${WORKING_DIR}/${DATASET}/global_features/DELG ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts \
1)kapture_image_retrieval_late_fusion
kapture_image_retrieval_late_fusion.py -v debug \ -i ${WORKING_DIR}/${DATASET}/map_plus_query/ \ --query ${WORKING_DIR}/${DATASET}/query/ \ -o ${WORKING_DIR}/${DATASET}/pairs_fusion.txt \ --topk ${TOPK} \ 'generalized_harmonic_mean' --weights 1 1 1 1`
Step3: using the pairs_fusion.txt obtained in step2, I did: `# 0a) Define paths and params PYTHONBIN=python3.8 WORKING_DIR=${PWD} DATASETS_PATH=${WORKING_DIR}/datasets DATASET=Aachen-Day-Night-v1.1 mkdir -p ${DATASETS_PATH}
TOPK=20 # number of retrieved images for mapping and localization KPTS=20000 # number of local features to extract
7) localization pipeline
LOCAL=r2d2_WASF_N8_big GLOBAL=Fusion kapture_pipeline_localize.py -v debug -f \ -s compute_image_pairs compute_matches geometric_verification \ -i ${WORKING_DIR}/${DATASET}/mapping \ --query ${WORKING_DIR}/${DATASET}/query \ -kpt ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/keypoints \ -desc ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/descriptors \ --pairsfile-path ${WORKING_DIR}/${DATASET}/pairs_fusion.txt \ -matches ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/NN_no_gv/matches \ -matches-gv ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/NN_colmap_gv/matches \ --colmap-map ${WORKING_DIR}/${DATASET}/colmap-sfm/${LOCAL}/Resnet101-AP-GeM-LM18 \ -o ${WORKING_DIR}/${DATASET}/colmap-localize/${LOCAL}/${GLOBAL} \ --topk ${TOPK} \ --config 2 `
additional questions:
1.how to correctly use the 'kapture_image_retrieval_late_fusion.py' in step2? I'm very confused especially with the -gfeat parameter; As shown above, what I passed to gfeat can't work properly so I use the default
Thanks in advance