Closed atillack closed 2 years ago
This PR introduces the analytical form of the rotation coefficient gradient correction to both OpenCL and Cuda.
The main driver for this change is numerical stability but it is also nice that it yields ~8% speedup per evaluation for Adadelta ;-)
Good to go :)
@diogomart Thank you! Merging.
This PR introduces the analytical form of the rotation coefficient gradient correction to both OpenCL and Cuda.
The main driver for this change is numerical stability but it is also nice that it yields ~8% speedup per evaluation for Adadelta ;-)