Why is small to lr? why is lr from small to big?

pengzhiliang / Conformer

Official code for Conformer: Local Features Coupling Global Representations for Visual Recognition

Apache License 2.0

531 stars 87 forks source link

Why is small to lr? why is lr from small to big? #6

Closed eeric closed 3 years ago

eeric commented 3 years ago

batch size=128, initial lr =0.001 Epoch: [0] [ 1070/14862] eta: 1:11:01 lr: 0.000001 loss_0: 5.6751 (5.6817) loss_1: 5.6846 (5.7059) time: 0.3053 data: 0.0002 max mem: 4992