search

amaralibey / Bag-of-Queries

BoQ: A Place is Worth a Bag of learnable Queries (CVPR 2024)
MIT License
96 stars 6 forks source link

For more details about the learning rate #10

Closed Li-Yun-star closed 1 month ago

Li-Yun-star commented 2 months ago

Thank you very much for your great work. Could you share the complete training code, especially the scheduling of the learning rate? For more details about the learning rate, I would like to know if it is immediately multiplied by 0.1 after the initial ten epochs warm up, or if it slowly decreases to the original 0.1 every ten epochs thereafter. My learning rate has changed as follows, but my training BoQ learning rate has changed as shown in the following figure, but it does not achieve better results. Can you provide some help? I would be very grateful.

lr

Li-Yun-star commented 2 months ago

This is another situation for learning rate scheduling. Which of these two situations is more suitable for your initial setup? lr_2

amaralibey commented 2 months ago

Hello @VClay,

Thanks for your interest in our work!

The learning rate is reduced by a factor of 0.1 right after the first 10 training epochs. We utilize torch.optim.lr_scheduler.MultiStepLR with a gamma value of 0.1. You can find the relevant code in the OpenVPRLab repository here (lines 102 to 123): https://github.com/amaralibey/OpenVPRLab/blob/main/src/core/vpr_framework.py

You can train BoQ (and many other VPR techniques) and play with their hyperparameters with OpenVPRLab, which is a framework that I recently launched to help develop and experiment with existing VPR techniques. Here is the link: https://github.com/amaralibey/OpenVPRLab

Additionally, I noticed that your learning rate is increasing from 0.0 to 4e-6, suggesting that your initial maximum learning rate is set to 4e-6. In our paper, we used 2e-4.

Please try training with OpenVPRLab, and let me know if that resolves the issue!

Li-Yun-star commented 2 months ago

Thank you for your great reply. Is the learning rate of the backbone also 2e-4? The framework you provided is excellent!

amaralibey commented 2 months ago

Hey @VClay, I am glad that you like the OpenVPRLab framework.

You can see in lines 85-88 (https://github.com/amaralibey/OpenVPRLab/blob/main/src/core/vpr_framework.py) that we are actually using the same learning rate for the backbone and the BoQ. But, I have also tried with 0.2*self.lr and 0.1*self.lr. They all achieved > 93% R@1 on MSLS-val.

Li-Yun-star commented 1 month ago

thanks