Open jrodriguezpuigvert opened 4 years ago
Thanks for sharing the error, and sorry for the late response. Could you share (the part of) your training script so that I can reproduce your error?
As the p.grad
is None according to your error message, I guess there's something wrong with the definition of the closure.
I found already the bug. In the forward function, you must use all parameters of the model otherwise some p.grad will be None after calling backward. That could be a problem if anyone would like to use it for fine-tunning. Other optimizers like Adam are more flexible in that case.
Wonderful! Many thanks. Kazuki can fix this issue then and close it.
Hello, I am experimenting with VOGN and I am getting this error:
Debugging I found that sometimes p.grad is None here
grads = [p.grad.data for p in params]
I didn't freeze any layer, I am learning from scratch.
Probably there is an error of my configuration. Do you have any hints about what could be the problem?
Here is the initialization of the optimizer: