search

linjieli222 / VQA_ReGAT

Research Code for ICCV 2019 paper "Relation-aware Graph Attention Network for Visual Question Answering"
https://arxiv.org/abs/1903.12314
MIT License
180 stars 38 forks source link

Learning rate related issues #28

Closed bobo123yhb closed 3 years ago

bobo123yhb commented 3 years ago

I am very interested in your work, but I have some doubts about the setting of the learning rate.I currently only have a 2080ti graphics card.How should the learning rate be adjusted?

linjieli222 commented 3 years ago

Do you mean that the batch size needs to be set smaller for a 2080ti GPU? My suggestion is to use a smaller learning rate and training longer until convergence. If the batch size is not much smaller than the default batch size provided, you may still use the same learning rate. If you have enough capacity to do a hyparameter search, I strongly suggest you to do so. As the default learning rate in my experiments are based on 4X 16GB V100 GPUs.