A couple ideas to try when get a test case that goes down,down,down,big up, down, down, down, big up, etc..
We already have dynamic step sizing that can be tweaked.
Stochastically reordering the hessian updates.
Adding a keyword triggering backsteps if the energies is higher than the one x steps ago. We know that with x=1, taking backward steps actually reduces overall, average performance. But perhaps with x=2 default, we could improve it.
A couple ideas to try when get a test case that goes down,down,down,big up, down, down, down, big up, etc.. We already have dynamic step sizing that can be tweaked.