STEPPER basis/geometry mismatch

[dmejiar]

I have stumbled upon this error message a couple of times now. Here are the input and output files for a case where the error appeared:

scan_ltls_symm_gena2s_forgit.log scan_ltls_symm_gena2s_forgit.nw.txt

However, I have similar inputs (different geometries/functionals/grids) where the error does not appear. Furthermore, if I use the same attached input but with DRIVER the error goes away.

[edoapra]

@dmejiar do you see any reason to use the stepper geometry optimizer instead of driver? Thanks

[dmejiar]

@edoapra the geometry is close to being a minimum but the driver optimizer tries to take huge steps. If line search is activated, the large step is taken for the line search and then a extremely small "new step" follows. I deactivated the line search and used a smaller trust radius, but still driver oscillates for some ~30 steps before actually start going in the right direction. For the inputs were stepper works, the optimization has always been well behaved.

At first, I thought some grid issues (due to SCAN) might be the source of the ill behavior, but as you can see in the input I'm now using rather large grids but the issues with driver are still there.

scan_ltls_symm_gena2s_driver.log.gz

[edoapra]

The initial gradient of your geometry optimization is not very small. I see the AO basis set is tzvpp. Have you tried smaller basis set (e.g. tzv or even svp), just to get a first estimate of the minimum, before switching to tzvpp?

[edoapra]

I would also suggest to tighten the convergence parameters with scan.

convergence energy 1d-8

[edoapra]

Here is a workaround for the the stepper failure. Please change the ao basis section to

basis "ao basis" spherical
# * library def2-tzvpp
 H library def2-tzvpp
 Mn library def2-tzvpp
 C library def2-tzvpp
 N library def2-tzvpp
end
Let me know if it works for you

[dmejiar]

The initial gradient of your geometry optimization is not very small. I see the AO basis set is tzvpp. Have you tried smaller basis set (e.g. tzv or even svp), just to get a first estimate of the minimum, before switching to tzvpp?

I, on purpose, moved the position of one atom, but the rest of the geometry was taken from a previous optimization with the Weigend Coulomb Fitting cd basis (which is essentially the same as without density fitting).

This third attempt with a different cd basis is because I obtained a different geometry in deMon2k using the cd basis specified the input, and I'm trying to see if I can find the same geometry with NWChem. So far, I haven't been able to.

[edoapra]

I am still not sure why a smaller basis set cannot be used ...

[dmejiar]

It can be used. All I wanted to say is that I already had an estimate of the minimum-obtained with the same ao basis but with different cd basis-but that the gradient was large because I displaced an atom on top of changing the cd basis.

By the way, the input with your workaround did the trick.

[edoapra]

@dmejiar I have done a few test on this case. What I found is that the cd basis set used in the input seemed to be the root cause for the numerical instability. Either turning off cd fitting or using a different cd basis (e.g. def2-jfit) gets you a numerically stable geometry optimization. issue_166.tar.gz

[dmejiar]

@edoapra wow, I wish I could have access that many cores without having to wait in queue for many days.

I think you are right about the cd basis. I was able to optimize the structure using the cd basis in the input with STEPPER, but the computation was not that smooth and the geometry is not as close to the result without cd fitting as with other cd basis. Thanks for looking into that.

I'm left wondering why the original error (basis/geometry mismatch) happened in the first place? The thing is that it didn't happened all the time, so could it be something wrong with the IO in my cluster?

[edoapra]

I'm left wondering why the original error (basis/geometry mismatch) happened in the first place? The thing is that it didn't happened all the time, so could it be something wrong with the IO in my cluster?

There is nothing wrong in your cluster. It's a bug in stepper. I do not see any reason to fix the bug, given that 1) there is a work-around and 2) I do not see any reason to use stepper instead of driver

[dmejiar]

Ok! Lastly, does lindep has an effect on the cd fitting basis?

[edoapra]

Ok! Lastly, does lindep has an effect on the cd fitting basis?

No, lindep has only effect for the AO basis (we do use Lowdin canonical orthogonalization)

[dmejiar]

Understood! I think you can close this now

[dmejiar]

@edoapra I did not know where to put this comment but, since it is related to the inputs you tested, I decided to comment here.

For a while, I have tried to do an optimization with density fitting and then switch it off with unset "basis:cd basis:*", but found always really large gradients for all the steps after the unsetting. In the input issue_166_nofit.nw you had a similar unset "cd basis". Either way works for single point calculations, but neither for optimizations (note that you never set a cd basis in your input, so everything was ok). I have found that for optimizations the additional unset dft:cdfit is needed, otherwise the nwdft module will add a contribution to the gradient from the unset cd basis. Perhaps this is something already documented, but I couldn't find any documentation about this.