Large time taken for feature matching between query and database images for pitts30k

[ur10]

Hello, I have been trying to reproduce the results as mentioned in the Patch-NetVLAD paper. I started with pitts30k datataset. I have followed the steps as mentioned in the repo for feature extraction and matching . However, during the feature matching stage while running with 4 RTX 2080 GPUs each of 12GB it says that the completion time is about 72 hours(~47s/itr). Is this expected behaviour or am I doing something wrong. My steps -

Feature extract

python feature_extract.py --config_path patchnetvlad/configs/performance.ini --dataset_file_path=pitts30k_imageNames_index.txt --dataset_root_dir=/home/ur10/data/pitts30k --output_features_dir patchnetvlad/output_features/pitts30k_index

python feature_extract.py --config_path patchnetvlad/configs/performance.ini -dataset_file_path=pitts30k_imageNames_query.txt --dataset_root_dir=/home/ur10/data/pitts30k --output_features_dir patchnetvlad/output_features/pitts30k_query

Feature Match

python feature_match.py --config_path patchnetvlad/configs/performance.ini --dataset_root_dir=/home/ur10/data/pitts30k --query_file_path=pitts30k_imageNames_query.txt --index_file_path=pitts30k_imageNames_index.txt --query_input_features_dir patchnetvlad/output_features/pitts30k_query --index_input_features_dir patchnetvlad/output_features/pitts30k_index --ground_truth_path patchnetvlad/dataset_gt_files/pitts30k_test.npz --result_save_folder patchnetvlad/results/pitts30k

I have taken care of the imageNames file so that I get 10k ref and 6.3k query images.

[Tobias-Fischer]

Hi, are you sure that the GPU is being used?

[ur10]

So, my performance.ini file looks something like this -

[feature_extract]
batchsize = 5
cachebatchsize = 5
imageresizew = 640
imageresizeh = 480

[feature_match]
matcher = RANSAC
n_values_all = 1,5,10,20,50,100
dataset_name = NA
pred_input_path = None
imageresizew = 640
imageresizeh = 480
patchweights2use = 0.45,0.15,0.4

[global_params]
pooling = patchnetvlad
resumepath = ./pretrained_models/pittsburgh_WPCA
threads = 16
num_pcs = 4096
ngpu = 4
patch_sizes = 2,5,8
strides = 1,1,1

and I have verified that I am not using the --nocuda option.

[ur10]

Hello @Tobias-Fischer , could you provide some hints as to why this might be happening? Also, what is the usual expected time for the feature matching process to get completed?

[Tobias-Fischer]

Hi @StephenHausler - could you please take a look here?

I have a feeling that you are trying to exhaustively match each query image to all reference images, instead of just reranking the top 100 matches provided by "vanilla" NetVLAD. Could this be the case @ur10?

[StephenHausler]

I've taken a look, first I'm not sure our code (at least in the matching stage) will actually use all four GPUs. Can you please try using just one GPU and test the speed?

Second, in a debugger, can you please verify that the variable "pred" on line 150 of local_matcher.py is length 100? And in debugger, also check to make sure all the variables qfeat and dbfeat are on the GPU.

The expected speed should be much faster than you are getting now, for the performance version, maybe 5 seconds per query image.