search

yaodongyu / TRADES

TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization)
MIT License
518 stars 123 forks source link

Bug in learning rate adjustment #11

Closed yaircarmon closed 5 years ago

yaircarmon commented 5 years ago

There seems to be a bug in the adjust_learning_rate function in train_trades_cifar10.py; it only decreases the learning rate once at epoch 75 (the code in the elif clauses is never reached).

yaodongyu commented 5 years ago

Thanks for proposing this.

Yes, the previous adjust_learning_rate function will only decay once. I fixed the bug in train_trades_cifar10.py and train_trades_mnist.py .

https://github.com/yaodongyu/TRADES/blob/c56d2f281112b4a003d0c25555e58a93fb070db9/train_trades_cifar10.py#L137

https://github.com/yaodongyu/TRADES/blob/c56d2f281112b4a003d0c25555e58a93fb070db9/train_trades_mnist.py#L132

Also, note that this will not affect the performance of our model, as we use early stopping after the first learning rate decay.