-
Congratulations on the new optimizer, excited to try it out! In our fastai discussion group it was brought up that your optimizer seems similar to HGD. Would you be able to summarize the key differenc…
-
Please replace state['previous_grad'] = grad with state['previous_grad'] = grad.clone() in line 117.
-
## 🚀 Feature
Add [diffGrad](https://arxiv.org/abs/1909.11015) optimizer to Pytorch.
-
This got a bit confusing post, next (following) comment below is minimal code to illustrate the issue...
I use a MLP according to the softmax.ipynb example as well as the winit function to initia…