search

mlcommons / training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes
https://mlcommons.org/en/groups/training
Apache License 2.0
92 stars 66 forks source link

Resnet Label Smoothing #365

Closed bitfort closed 4 years ago

bitfort commented 4 years ago

We should clarify this as it was added in v0.6.

bitfort commented 4 years ago

SWG:

Label smoothing for resnet It is reportedly set to 0 in the reference (AI: check) but previous submissions from both Google and NVIDIA set it to 0.1. We believe label smoothing was changed to be 0.1 in v0.6 but the reference was possibly never updated to reflect this. If you do not use label smoothing, we believe it is very hard to get convergence and thus you may not be able.

We would allow people to move to 0.1 during review if they did not use it.

AI(We will check the reference)