This PR adds pytorch implementation of Cocob Optimizer, introduced in this paper. Feature request to add in pytorch/contrib is here.
Original Tensorflow implementation is here and this PR is based on this repo.
Apart of adding COCOB and it's tests, there is a small code refactor. A lot of code in TestSWA wasn't specific to testing SWA optimizer. So I have plucked that out into a general class TestOptim, and made TestSWA to inherit from TestOptim. TestCocob also inherits TestOptim to avoid code duplication.
This PR adds pytorch implementation of Cocob Optimizer, introduced in this paper. Feature request to add in pytorch/contrib is here.
Original Tensorflow implementation is here and this PR is based on this repo.
Apart of adding COCOB and it's tests, there is a small code refactor. A lot of code in
TestSWAwasn't specific to testingSWAoptimizer. So I have plucked that out into a general classTestOptim, and madeTestSWAto inherit fromTestOptim.TestCocobalso inheritsTestOptimto avoid code duplication.