Closed rlouf closed 4 years ago
Optimizers may be needed during warmup, but it is also fun to show they can be written as kernels. This PR adds Adam and RMSProp.
Good job me :+1: merging.
Optimizers may be needed during warmup, but it is also fun to show they can be written as kernels. This PR adds Adam and RMSProp.