Sometimes the NR method struggles to converge the "final result", instead it bounces around it. This is particularly problematic when convection extends from the surface, which is expected to be common for many systems.
Since the NR method profiles a descent direction x_dif, we can scale this vector to obtain an optimal step size by using a line search method. I've already found that scaling this factor by 0.1x works well to improve model performance in some cases.
This should be relatively simple to implement:
Start at x = x_old with residuals r = r_old
Calculate NR step, providing x_dif and r_tst residuals
Sample r_tst for various x = x_old + a * x_dif, where a is varied within the semi-closed interval (0,1].
Select the best value of a (and thus the new x) based on the optimal r_tst.
We could probably just use a handful of a values at each iteration, so the performance change from this should be minimal.
Sometimes the NR method struggles to converge the "final result", instead it bounces around it. This is particularly problematic when convection extends from the surface, which is expected to be common for many systems.
Since the NR method profiles a descent direction
x_dif, we can scale this vector to obtain an optimal step size by using a line search method. I've already found that scaling this factor by 0.1x works well to improve model performance in some cases.This should be relatively simple to implement:
x = x_oldwith residualsr = r_oldx_difandr_tstresidualsr_tstfor variousx = x_old + a * x_dif, whereais varied within the semi-closed interval(0,1].a(and thus the newx) based on the optimalr_tst. We could probably just use a handful ofavalues at each iteration, so the performance change from this should be minimal.