"Transformation did not converge". Units?

[andersx]

Hey, finally got time to write some code (see simple example below). I get the error "Transformation did not converge in 20 iterations", already at the first optimization step.

https://github.com/azag0/pyberny/blob/master/berny/coords.py#L327

If I add print(Math.rms(cur.dcart)), I see that the transformation diverges:

0.000608133218922
0.000329078920598
0.000347330407485
0.000376978957967
0.000417930251795
0.000471971184474
0.000542455016937
0.000634742580873
0.000757064648142
0.000921799269649
0.00114727178085
0.00146031116854
0.00190008914962
0.00252452439484
0.00342280156923
0.00474552800249
0.00679402489798
0.0103050015856
0.0169683896358
0.0270444544195

I think this might simply be a unit conversion issue. The energy/gradients are in hartree and hartree/bohr, respectively. I tried giving the input geometry in both bohr and angstrom. I assume this is important for building the correct internal coordinates. In which units should the input coordinates, energies, and gradients be parsed? Should input xyz files be in angstrom? What units are the geom.coords array in?

Thanks! Anders

geom = geomlib.readfile('ethanol.xyz')
optimizer = berny.Berny(geom)

for geom in optimizer:
    energy, gradients = pm6_gradient(geom.coords, geom.species)
    info = optimizer.send((energy, gradients))

[andersx]

Full output from ten first iterations (the optimization doesn't convert, so I truncated the output):

0 Internal coordinates:
0 * Number of fragments: 1
0 * Number of internal coordinates: 33
0 * Number of strong bonds: 8
0 * Number of strong angles: 13
0 * Number of strong dihedrals: 12
1 Energy: 1.41500482861
1 Minimization on sphere was performed:
1 * Trust radius: 0.3
1 * Number of negative eigenvalues: 0
1 * Lowest eigenvalue: 0.00541
1 * lambda: -0.209
1 Quadratic step: RMS: 0.0522, max: 0.155
1 * Predicted energy change: -0.0401
1 Total step: RMS: 0.0522, max: 0.155
1 Transformation did not converge in 20 iterations
1 * RMS(dcart): 0.0359, RMS(dq): 0.00188
1 Convergence criteria:
1 * Gradient RMS: 0.045 > 0.0003 => no
1 * Gradient maximum: 0.11 > 0.00045 => no
1 * Step RMS: 0.0523 > 0.0012 => no
1 * Step maximum: 0.155 > 0.0018 => no
2 Energy: 1.52273315166
2 Hessian update information:
2 * Change: RMS: 0.0111, max: 0.0748
2 Trust update: Fletcher's parameter: -2.69
2 Linear interpolation:
2 * Energies: 1.5227332, 1.4150048
2 * Derivatives: 0.0507, 0.0614
2 * Cubic interpolation was performed: t = 0.934
2 * Interpolated energy: 1.4130227
2 Minimization on sphere was performed:
2 * Trust radius: 0.075
2 * Number of negative eigenvalues: 0
2 * Lowest eigenvalue: 0.00541
2 * lambda: -2.58
2 Quadratic step: RMS: 0.0131, max: 0.0389
2 * Predicted energy change: -0.0148
2 Total step: RMS: 0.0358, max: 0.106
2 Transformation did not converge in 20 iterations
2 * RMS(dcart): 0.0255, RMS(dq): 0.000326
2 Convergence criteria:
2 * Gradient RMS: 0.0377 > 0.0003 => no
2 * Gradient maximum: 0.0785 > 0.00045 => no
2 * Step RMS: 0.0357 > 0.0012 => no
2 * Step maximum: 0.106 > 0.0018 => no
3 Energy: 1.45106143328
3 Hessian update information:
3 * Change: RMS: 0.00277, max: 0.0663
3 Trust update: Fletcher's parameter: 4.83
3 Linear interpolation:
3 * Energies: 1.4510614, 1.4150048
3 * Derivatives: 0.0187, 0.0195
3 * Cubic interpolation was performed: t = 0.937
3 * Interpolated energy: 1.4144096
3 Minimization on sphere was performed:
3 * Trust radius: 0.15
3 * Number of negative eigenvalues: 0
3 * Lowest eigenvalue: 0.00541
3 * lambda: -1.28
3 Quadratic step: RMS: 0.0261, max: 0.0798
3 * Predicted energy change: -0.0297
3 Total step: RMS: 0.0106, max: 0.0336
3 Transformation did not converge in 20 iterations
3 * RMS(dcart): 0.0073, RMS(dq): 0.000543
3 Convergence criteria:
3 * Gradient RMS: 0.0434 > 0.0003 => no
3 * Gradient maximum: 0.105 > 0.00045 => no
3 * Step RMS: 0.0106 > 0.0012 => no
3 * Step maximum: 0.0336 > 0.0018 => no
4 Energy: 1.4734181766
4 Hessian update information:
4 * Change: RMS: 0.000139, max: 0.00105
4 Trust update: Fletcher's parameter: -0.753
4 Linear interpolation:
4 * Energies: 1.4734182, 1.4150048
4 * Derivatives: 0.0298, 0.0319
4 * Cubic interpolation was performed: t = 0.937
4 * Interpolated energy: 1.4140159
4 Minimization on sphere was performed:
4 * Trust radius: 0.037
4 * Number of negative eigenvalues: 0
4 * Lowest eigenvalue: 0.00541
4 * lambda: -5.35
4 Quadratic step: RMS: 0.00653, max: 0.0201
4 * Predicted energy change: -0.00759
4 Total step: RMS: 0.0189, max: 0.0575
4 Transformation did not converge in 20 iterations
4 * RMS(dcart): 0.0127, RMS(dq): 0.000146
4 Convergence criteria:
4 * Gradient RMS: 0.0422 > 0.0003 => no
4 * Gradient maximum: 0.102 > 0.00045 => no
4 * Step RMS: 0.0189 > 0.0012 => no
4 * Step maximum: 0.0575 > 0.0018 => no
5 Energy: 1.43316895348
5 Hessian update information:
5 * Change: RMS: 0.000438, max: 0.00953
5 Trust update: Fletcher's parameter: 5.31
5 Linear interpolation:
5 * Energies: 1.433169, 1.4150048
5 * Derivatives: 0.00952, 0.00971
5 * Cubic interpolation was performed: t = 0.938
5 * Interpolated energy: 1.4147105
5 Minimization on sphere was performed:
5 * Trust radius: 0.075
5 * Number of negative eigenvalues: 0
5 * Lowest eigenvalue: 0.00541
5 * lambda: -2.64
5 Quadratic step: RMS: 0.0131, max: 0.0403
5 * Predicted energy change: -0.0151
5 Total step: RMS: 0.00532, max: 0.0165
5 Transformation did not converge in 20 iterations
5 * RMS(dcart): 0.00365, RMS(dq): 0.000288
5 Convergence criteria:
5 * Gradient RMS: 0.0442 > 0.0003 => no
5 * Gradient maximum: 0.108 > 0.00045 => no
5 * Step RMS: 0.00532 > 0.0012 => no
5 * Step maximum: 0.0165 > 0.0018 => no
6 Energy: 1.44468762251
6 Hessian update information:
6 * Change: RMS: 2.33e-05, max: 0.000296
6 Trust update: Fletcher's parameter: -0.763
6 Linear interpolation:
6 * Energies: 1.4446876, 1.4150048
6 * Derivatives: 0.0155, 0.016
6 * Cubic interpolation was performed: t = 0.938
6 * Interpolated energy: 1.4145181
6 Minimization on sphere was performed:
6 * Trust radius: 0.019
6 * Number of negative eigenvalues: 0
6 * Lowest eigenvalue: 0.00541
6 * lambda: -10.8
6 Quadratic step: RMS: 0.00326, max: 0.0101
6 * Predicted energy change: -0.00382
6 Total step: RMS: 0.00946, max: 0.0292
6 Transformation did not converge in 20 iterations
6 * RMS(dcart): 0.00637, RMS(dq): 0.000199
6 Convergence criteria:
6 * Gradient RMS: 0.0436 > 0.0003 => no
6 * Gradient maximum: 0.106 > 0.00045 => no
6 * Step RMS: 0.00946 > 0.0012 => no
6 * Step maximum: 0.0292 > 0.0018 => no
7 Energy: 1.42409944065
7 Hessian update information:
7 * Change: RMS: 0.000115, max: 0.00224
7 Trust update: Fletcher's parameter: 5.39
7 Linear interpolation:
7 * Energies: 1.4240994, 1.4150048
7 * Derivatives: 0.00479, 0.00484
7 * Cubic interpolation was performed: t = 0.938
7 * Interpolated energy: 1.4148588
7 Minimization on sphere was performed:
7 * Trust radius: 0.037
7 * Number of negative eigenvalues: 0
7 * Lowest eigenvalue: 0.00541
7 * lambda: -5.37
7 Quadratic step: RMS: 0.00653, max: 0.0202
7 * Predicted energy change: -0.00762
7 Total step: RMS: 0.00268, max: 0.00826
7 Transformation did not converge in 20 iterations
7 * RMS(dcart): 0.00183, RMS(dq): 0.000386
7 Convergence criteria:
7 * Gradient RMS: 0.0446 > 0.0003 => no
7 * Gradient maximum: 0.109 > 0.00045 => no
7 * Step RMS: 0.00268 > 0.0012 => no
7 * Step maximum: 0.00826 > 0.0018 => no
8 Energy: 1.42996129145
8 Hessian update information:
8 * Change: RMS: 1.95e-05, max: 0.000371
8 Trust update: Fletcher's parameter: -0.77
8 Linear interpolation:
8 * Energies: 1.4299613, 1.4150048
8 * Derivatives: 0.00785, 0.00798
8 * Cubic interpolation was performed: t = 0.938
8 * Interpolated energy: 1.4147633
8 Minimization on sphere was performed:
8 * Trust radius: 0.0094
8 * Number of negative eigenvalues: 0
8 * Lowest eigenvalue: 0.00541
8 * lambda: -21.7
8 Quadratic step: RMS: 0.00163, max: 0.00504
8 * Predicted energy change: -0.00191
8 Total step: RMS: 0.00474, max: 0.0146
8 Transformation did not converge in 20 iterations
8 * RMS(dcart): 0.0032, RMS(dq): 0.000688
8 Convergence criteria:
8 * Gradient RMS: 0.0443 > 0.0003 => no
8 * Gradient maximum: 0.108 > 0.00045 => no
8 * Step RMS: 0.00474 > 0.0012 => no
8 * Step maximum: 0.0146 > 0.0018 => no
9 Energy: 1.41955565648
9 Hessian update information:
9 * Change: RMS: 9.08e-05, max: 0.000685
9 Trust update: Fletcher's parameter: 5.44
9 Linear interpolation:
9 * Energies: 1.4195557, 1.4150048
9 * Derivatives: 0.0024, 0.00241
9 * Cubic interpolation was performed: t = 0.938
9 * Interpolated energy: 1.4149321
9 Minimization on sphere was performed:
9 * Trust radius: 0.019
9 * Number of negative eigenvalues: 0
9 * Lowest eigenvalue: 0.0054
9 * lambda: -10.8
9 Quadratic step: RMS: 0.00326, max: 0.0101
9 * Predicted energy change: -0.00382
9 Total step: RMS: 0.0015, max: 0.00414
9 Transformation did not converge in 20 iterations
9 * RMS(dcart): 0.000965, RMS(dq): 0.00132
9 Convergence criteria:
9 * Gradient RMS: 0.0448 > 0.0003 => no
9 * Gradient maximum: 0.109 > 0.00045 => no
9 * Step RMS: 0.00149 > 0.0012 => no
9 * Step maximum: 0.00414 > 0.0018 => no
10 Energy: 1.42251250179
10 Hessian update information:
10 * Change: RMS: 1.96e-05, max: 0.000245
10 Trust update: Fletcher's parameter: -0.773
10 Linear interpolation:
10 * Energies: 1.4225125, 1.4150048
10 * Derivatives: 0.00396, 0.00399
10 * Cubic interpolation was performed: t = 0.938
10 * Interpolated energy: 1.4148845
10 Minimization on sphere was performed:
10 * Trust radius: 0.0047
10 * Number of negative eigenvalues: 0
10 * Lowest eigenvalue: 0.0054
10 * lambda: -43.6
10 Quadratic step: RMS: 0.000816, max: 0.00252
10 * Predicted energy change: -0.000958
10 Total step: RMS: 0.00269, max: 0.00731
10 Transformation did not converge in 20 iterations
10 * RMS(dcart): 0.0017, RMS(dq): 0.00253
10 Convergence criteria:
10 * Gradient RMS: 0.0447 > 0.0003 => no
10 * Gradient maximum: 0.109 > 0.00045 => no
10 * Step RMS: 0.00267 > 0.0012 => no
10 * Step maximum: 0.0073 > 0.0018 => no
...

[jhrmnn]

The fact that the transformation from internal to Cartesian coordinates does not converge (and even diverges) is actually fine and happens often. This is the result of the overdetermination of the system by the redundant internal coordinates. When it happens, only the first iteration is considered.

But there actually was a trivial bug now fixed in 5f445122664ab9b3db1246d9d94f45c478363e0d that might have caused that the geometry did not converge. I tried to optimize an experimental ethanol geometry with PBE/6-31G and it converges in 6 steps. Could you try again with your geometry and PM6?

Other than that, the energy can be in whatever units U and the gradients must be in U/bohr. As for geometry, the units are specified by the format, so it would be angstroms for XYZ files.

Btw, how do you get PM6 in Python?

[andersx]

Thanks, I think correct units might have been the culprit. If I convert energies and gradients to atomic units before I parse them to the optimizer I get convergence in 13 steps for the ethanol molecule but eventually the optimization will diverge if I use kcal/mol as energy unit (I guess it also affects the convergence criteria).

Btw, I am just using a super simple simple wrapper, here is one I use for OM2 gradients with MNDO99: https://gist.github.com/andersx/a6ec99562994757095ff1a7b549b08c8

[jhrmnn]

Uhm, yes actually, the convergence criteria have units, Ha/bohr, so it's not any energy unit U.

Can I close this?

I'm looking for something that would give me fast, realistic gradients as a black box for testing purposes. Ideally it would be installable with Pip. I'm using pyscf at the moment, but that still requires compilation step. Any suggestions are welcome...

[andersx]

Sure, go ahead and close this!

In principle one could write a wrapper for MOPAC 7.1 which is public domain, and distribute a python the code+interface via PyPI. That is the best I can think of right now.