The training top-1 accuracy is only 80% in cifar100. Did we make a mistake or miss something?

[zkf331]

Train ZenNet-2.0M in CIFAR100

Test model： zennet_cifar100_model_size2M_res32

Data augmentation：

• subtracting the channel mean and dividing the channel standard deviation
• mixup
• label-smoothing
• random erasing
• random crop/resize/flip/lightting
• Auto Augment

Train optimizer：

• SGD optimizer with momentum 0.9
• weight decay 5e-4 for CIFAR10/100
• Learning rate 0.1 with batch size 256
• Cosine learning rate decay
• 1440 epochs in CIFAR10/100

Hi, your work is exciting and inspires us a lot. So we try to reproduce，but we train the test model according to the above configuration, and the training top-1 accuracy is only 80% in cifar100. Did we make a mistake or miss something?

[MingLin-home]

You configuration seems to be correct. You should be able to achieve 83%~84% easily under this setting. We will release training code after publication.

[MingLin-home]

@zkf331

We have identified your problem. Please use batch size 64 and only single GPU to train this model. When using larger batch size such as 256, the accuracy drops. We doubt this is related to some local minima issue when using large batch size.