pengzhiliang
commented
3 years ago I'm sorry that something wrong has happened in your experiment.
On the ImageNet1k, if using conformer_small_path16, the log is:
{"train_lr": 1.000000000000014e-06, "train_loss_0": 3.479622867107391, "train_loss_1": 3.475991682386398, "test_loss": 6.931091730831233, "test_loss_0": 3.4690902396922803, "test_loss_1": 3.4620015056988667, "test_acc1": 0.1160000015258789, "test_acc1_head1": 0.11200000076293945, "test_acc1_head2": 0.1640000033569336, "epoch": 0, "n_parameters": 37673424} {"train_lr": 0.001600200000000024, "train_loss_0": 2.916393296575546, "train_loss_1": 3.0137832653045655, "test_loss": 4.195239642194209, "test_loss_0": 2.0028400967139324, "test_loss_1": 2.192399521820418, "test_acc1": 22.068000575561523, "test_acc1_head1": 21.402000715332033, "test_acc1_head2": 17.656000471191405, "epoch": 5, "n_parameters": 37673424}
And I notice that the 'n_parameters' in your log is 77.5M, which is neither conformer_small parameter nor a parameter of conformer_base
{"train_lr": 0.0006003999999998758, "train_loss_0": 5.2468776138196835, "train_loss_1": 5.2314488993825545, "test_loss": 12.697948946271623, "test_loss_0": 6.487931878226144, "test_loss_1": 6.210017108917237, "test_acc1": 0.0, "test_acc1_head1": 0.0, "test_acc1_head2": 0.0, "epoch": 4, "n_parameters": 77557992}
So, have you changed the network or dataset?
eeric
{"train_lr": 0.0006003999999998758, "train_loss_0": 5.2468776138196835, "train_loss_1": 5.2314488993825545, "test_loss": 12.697948946271623, "test_loss_0": 6.487931878226144, "test_loss_1": 6.210017108917237, "test_acc1": 0.0, "test_acc1_head1": 0.0, "test_acc1_head2": 0.0, "epoch": 4, "n_parameters": 77557992}