blackfeather-wang / ISDA-for-Deep-Networks

An efficient implicit semantic augmentation method, complementary to existing non-semantic techniques.
582 stars 93 forks source link

Performance improvements on ImageNet #4

Closed JamesHao-ml closed 4 years ago

JamesHao-ml commented 4 years ago

I found that you update the Arxiv paper on Aril 25 and the performance on ImageNet is largely improved, from 0.28% gain to 1.1% gain on ResNet-50. Is there any special change that leads to the larger improvements? Thanks.

blackfeather-wang commented 4 years ago

Thank you for your question. In the original conference version, we train ResNet-50 for 120 epochs with a step lr schedule (*0.1 at 30, 60, 90 epoch). The new results use a longer schedule with 300 epochs and a cosine lr schedule, following other sota augmentation techniques. We find that these regularization techniques generally tend to be more effective with longer training time.

JamesHao-ml commented 4 years ago

Thanks!