Fix VGG19 and refactor regularized problems

fsschneider / DeepOBS

DeepOBS: A Deep Learning Optimizer Benchmark Suite

MIT License

103 stars 34 forks source link

Fix VGG19 and refactor regularized problems #23

Closed f-dangel closed 4 years ago

f-dangel commented 4 years ago

The forward pass for VGG19 was not working on CIFAR10, CIFAR100. I added tests that fail before and pass after the fix
Test problems with regularization penalize only the weights and not the bias terms. Hoisted the implementation to create the parameter groups by implementing a new test problem class
@fsschneider: I was not sure about regularization for the WRNs. We should discuss and refactor them as well.

fsschneider commented 4 years ago

Oh wow, I messed something up there. Thanks. Originally the VGG networks were working and passed my tests. The bug was introduced with the "apply black formatting" commit, which should not happen.

Tomorrow, I will look through the changes and merge them.

Thanks again for finding and fixing them.

fsschneider commented 4 years ago

I'll merge it into this new branch, were I will re-write and re-factor the tests (and possibly other stuff)