search

gouthamvgk / SuperGlue_training

This repository contains the training code for SuperGlue. Uses COCO dataset by generating random homographies.
Other
101 stars 22 forks source link

Evaluation recall can't reach 96.22 on COCO2017 test set #7

Closed kimivital closed 2 years ago

kimivital commented 2 years ago

The recall of the model I trained with the default yaml file is 10% lower than COCO_homo, are you using any pretrained weights for training COCO_homo? How to reach the recall of COCO_homo? Thanks.

Homography using RANSAC Indoor test set (Mean over 144 pairs) AUC@5 AUC@10 AUC@25 Prec Recall
28.79 43.65 58.84 70.98 78.67
Outdoor test set (Mean over 145 pairs) AUC@5 AUC@10 AUC@25 Prec Recall
27.33 36.92 50.58 54.14 63.94
COCO2017 test set (Mean over 199 pairs) AUC@5 AUC@10 AUC@25 Prec Recall
34.30 51.15 70.66 83.28 87.83
szgy66 commented 2 years ago

I enev can't reach your test leval, what happend？

gouthamvgk commented 2 years ago

@kimivital I am not using any pretrained model. How many epochs did you train for?

szgy66 commented 2 years ago

The numbers of coco train images are 118287, the batch size is 4, the epochs are 10, is it suitable?

kimivital commented 2 years ago

@kimivital I am not using any pretrained model. How many epochs did you train for?

90 epochs, I adjusted pos_loss_weight to 0.9, and then I can achieve the following performance(at 27/90 epochs)

Homography using RANSAC Indoor test set (Mean over 144 pairs) AUC@5 AUC@10 AUC@25 Prec Recall
28.53 41.72 58.17 70.28 83.59
Outdoor test set (Mean over 145 pairs) AUC@5 AUC@10 AUC@25 Prec Recall
27.46 35.86 47.93 57.38 67.52
COCO2017 test set (Mean over 199 pairs) AUC@5 AUC@10 AUC@25 Prec Recall
32.41 51.75 72.41 82.66 94.21

There is still a small gap between the performance you gave

gouthamvgk commented 2 years ago

The numbers of coco train images are 118287, the batch size is 4, the epochs are 10, is it suitable?

You need to train for more epochs

gouthamvgk commented 2 years ago

@kimivital I am not using any pretrained model. How many epochs did you train for?

90 epochs, I adjusted pos_loss_weight to 0.9, and then I can achieve the following performance(at 27/90 epochs)

Homography using RANSAC Indoor test set (Mean over 144 pairs) AUC@5 AUC@10 AUC@25 Prec Recall 28.53 41.72 58.17 70.28 83.59

Outdoor test set (Mean over 145 pairs) AUC@5 AUC@10 AUC@25 Prec Recall 27.46 35.86 47.93 57.38 67.52

COCO2017 test set (Mean over 199 pairs) AUC@5 AUC@10 AUC@25 Prec Recall 32.41 51.75 72.41 82.66 94.21

There is still a small gap between the performance you gave

The released model was trained for 60 epochs. So it might improve with more training

kimivital commented 2 years ago

@kimivital I am not using any pretrained model. How many epochs did you train for?

90 epochs, I adjusted pos_loss_weight to 0.9, and then I can achieve the following performance(at 27/90 epochs) Homography using RANSAC Indoor test set (Mean over 144 pairs) AUC@5 AUC@10 AUC@25 Prec Recall 28.53 41.72 58.17 70.28 83.59 Outdoor test set (Mean over 145 pairs) AUC@5 AUC@10 AUC@25 Prec Recall 27.46 35.86 47.93 57.38 67.52 COCO2017 test set (Mean over 199 pairs) AUC@5 AUC@10 AUC@25 Prec Recall 32.41 51.75 72.41 82.66 94.21 There is still a small gap between the performance you gave

The released model was trained for 60 epochs. So it might improve with more training

Thanks for the reply, I trained 90 epochs, and the best performance was at the 27th. It may be some other random factor that causes the performance difference.