search

admb-project / admb

AD Model Builder
http://admb-project.org
Other
64 stars 19 forks source link

Merge Issue #160: Adding the capability to do single Newton steps using the inverse Hessian #182

Closed johnoel closed 3 years ago

johnoel commented 3 years ago

Copied initially from https://github.com/Cole-Monnahan-NOAA/admb/tree/add_hess_step

Cole-Monnahan-NOAA commented 3 years ago

@johnoel This reminds me I should update the console output to give some more information. Is it too late to do that or can I update my fork/branch still?

johnoel commented 3 years ago

Yes, you can update.

johnoel commented 3 years ago

Okay @Cole-Monnahan-NOAA, I forgot about inactive parameters in phases. Changes in commit 23c80b9 will check if the parameter is active. Please test and let me know.

Cole-Monnahan-NOAA commented 3 years ago

@johnoel Here's the output:

Par MGparm[5] : original grad=8.5597e-06 and updated grad= -0.000144485
Par MGparm[6] : original grad=3.53257e-06 and updated grad= -7.67114e-05
Par MGparm[13] : original grad=2.39255e-05 and updated grad= 0.000635574
Par MGparm[17] : original grad=5.53279e-06 and updated grad= -0.000175291
Par MGparm[18] : original grad=2.60588e-06 and updated grad= -0.0001207
Par SR_parm[1] : original grad=-0.00024033 and updated grad= -0.0237716
Par recdev1 index 1 : original grad=3.21513e-06 and updated grad= 0.000380508
Par recdev1 index 2 : original grad=3.21513e-06 and updated grad= 0.000380508
[truncated]
Par recdev1 index 39 : original grad=3.21513e-06 and updated grad= 0.000380508
Par selparm[232] : original grad=4.61444e-06 and updated grad= 0.000375142
Par selparm[234] : original grad=4.68196e-06 and updated grad= 0.000329463
Par selparm[236] : original grad=3.7379e-06 and updated grad= 0.000102783

So this is definitely closer but it's the same grads for the recdev1 parameters. This is because the ++jj needs to up a line. I implemented a fix in this commit and now the output is:

MGparm[5]: original grad=8.5597e-06 and updated grad=-0.000144485
MGparm[6]: original grad=3.53257e-06 and updated grad=-7.67114e-05
MGparm[13]: original grad=2.39255e-05 and updated grad=0.000635574
MGparm[17]: original grad=5.53279e-06 and updated grad=-0.000175291
MGparm[18]: original grad=2.60588e-06 and updated grad=-0.0001207
SR_parm[1]: original grad=-0.00024033 and updated grad=-0.0237716
recdev1(1): original grad=3.21513e-06 and updated grad=0.000380508
recdev1(2): original grad=4.61444e-06 and updated grad=0.000375142
[truncated]
recdev1(39): original grad=3.6569e-06 and updated grad=0.000428114
selparm[232]: original grad=-7.10296e-06 and updated grad=-0.00115854
selparm[234]: original grad=-1.26791e-05 and updated grad=-0.00190528
selparm[236]: original grad=-2.02016e-05 and updated grad=-0.0028014

Experimental feature -hess_step resulted in worse gradients (above).
Consider reoptimizing model to reset files. This suggests that the
the negative log-likelihood is not quadratic at the mode, or that the
Hessian does not approximate it well. In short, it suggests the model is
not converged. Check model structure and other output files to investigate
potential causes and solutions.

I also updated the console output and think it's much better. I can do a pull request to get these changes into the main issue160 branch on your repo if you want. Thoughts?

johnoel commented 3 years ago

Thanks @Cole-Monnahan-NOAA, push changes to issue160 branch. Yep, the ++jj was in the wrong line, thanks for catching that.