Hello, from the information in the model zoo for GQA, the learning rates of butd and ban-4 are 2e-4. However, in the file of configs/gqa/butd and configs/gqa/ban-4. the setting of learning rates is 2e-3. I want to know which learning rate is right for the models (butd, ban-4, ban-8) in the model zoo.
Thank you!
Hello, from the information in the model zoo for GQA, the learning rates of butd and ban-4 are 2e-4. However, in the file of configs/gqa/butd and configs/gqa/ban-4. the setting of learning rates is 2e-3. I want to know which learning rate is right for the models (butd, ban-4, ban-8) in the model zoo. Thank you!