Incompatibility with PySCF >= 2.5.0

[alexfleury-sb]

Incompatibility with PySCF >= 2.5.0

This error currently occurs in a single test: ansatz_generator.tests.test_uccsd.UCCSDTest.test_uccsd_H4_open, where unrestricted MP2 amplitudes are computed as initial parameters for a UCCSD ansatz in VQE.

With PySCF<=2.4.0, the test passes. However, with the latest versions of PySCF (2.5.0 and on) the test currently returns an error: it seems there was a change in a the eri data-structures. It may be that a simple fix could be done inside our MP2Solver class.

Expected Behavior Find a way to make this test pass for both PySCF 2.5.0 while still supporting the older versions. Test locally in both environment (you can easily switch between versions with pip install and uninstall, by specifying the desired version). The continuous integration environment yml file should then be bumped to pyscf latest version so that all automated tests run with it on.

Current Behavior An IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed is raised when computing MP2 initial parameters for open-shell systems.

Steps to Reproduce (minimal example)

Minimal script:

from tangelo.molecule_library import mol_H4_cation_sto3g
from tangelo.toolboxes.ansatz_generator.uccsd import UCCSD

uccsd_ansatz = UCCSD(mol_H4_cation_sto3g)
uccsd_ansatz.build_circuit()

outputs

Traceback (most recent call last):
  File "/Users/alexandre.fleury/Work/Scratch/sandbox.py", line 6, in <module>
    uccsd_ansatz.build_circuit()
  File "/Users/alexandre.fleury/Repos/Tangelo/tangelo/toolboxes/ansatz_generator/uccsd.py", line 164, in build_circuit
    self.set_var_params()
  File "/Users/alexandre.fleury/Repos/Tangelo/tangelo/toolboxes/ansatz_generator/uccsd.py", line 127, in set_var_params
    initial_var_params = self._compute_mp2_params()
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexandre.fleury/Repos/Tangelo/tangelo/toolboxes/ansatz_generator/uccsd.py", line 283, in _compute_mp2_params
    mp2.simulate()
  File "/Users/alexandre.fleury/Repos/Tangelo/tangelo/algorithms/classical/mp2_solver.py", line 316, in simulate
    return self.solver.simulate()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexandre.fleury/Repos/Tangelo/tangelo/algorithms/classical/mp2_solver.py", line 89, in simulate
    _, self.mp2_t2 = self.mp2_fragment.kernel()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexandre.fleury/.pyenv/versions/qsdk/lib/python3.11/site-packages/pyscf/mp/mp2.py", line 598, in kernel
    eris = self.ao2mo(mo_coeff)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexandre.fleury/.pyenv/versions/qsdk/lib/python3.11/site-packages/pyscf/mp/ump2.py", line 437, in ao2mo
    return _make_eris(self, mo_coeff, verbose=self.verbose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexandre.fleury/.pyenv/versions/qsdk/lib/python3.11/site-packages/pyscf/mp/ump2.py", line 499, in _make_eris
    eris._common_init_(mp, mo_coeff)
  File "/Users/alexandre.fleury/.pyenv/versions/qsdk/lib/python3.11/site-packages/pyscf/mp/ump2.py", line 475, in _common_init_
    mo_a = mo_coeff[0][:,mo_idx[0]]
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

The IndexError points out the fact that the mo_coeff data structures have changed from PySCF 2.4.0 to 2.5.0. The solution as of now is to restrict the PySCF support to 2.4.0 and older versions, but it would be great if this bug could be fixed for future releases.

[anushkrishnav]

Let me take a look into this, will ask for more info as i go about it

[alexfleury-sb]

Thank you! We are looking for your solution, and are able to provide as much info as you need.

[anushkrishnav]

Awesome I will send my first pr today

[anushkrishnav]

Seems like self.mp2_fragment = self.mp.RMP2(self.mean_field, frozen=self.frozen) from simulate() returns different mo_coeff for the versions

https://github.com/goodchemistryco/Tangelo/blob/6791dc83d9dac7ee5aad9d5a1f3dcbed81aac2cb/tangelo/algorithms/classical/mp2_solver.py#L86

2.5 has a mo_coeff -> [[ 0.41972471 -0.47196477 -0.3961148 -0.96961923] [ 0.41972471 -0.47196477 0.3961148 0.96961923] [ 0.28346875 0.6196045 0.67216556 -0.56718079] [ 0.28346875 0.6196045 -0.67216556 0.56718079]]

while 2.4 has mo_coeff [[[ 0.41972471 -0.47196477 -0.3961148 -0.96961923] [ 0.41972471 -0.47196477 0.3961148 0.96961923] [ 0.28346875 0.6196045 0.67216556 -0.56718079] [ 0.28346875 0.6196045 -0.67216556 0.56718079]]

[[ 0.41972471 -0.47196477 -0.3961148 -0.96961923] [ 0.41972471 -0.47196477 0.3961148 0.96961923] [ 0.28346875 0.6196045 0.67216556 -0.56718079] [ 0.28346875 0.6196045 -0.67216556 0.56718079]]]

i will investigate it further to find a fix but this is the root of the problem

[anushkrishnav]

OKay after close to an hr of looking in 2.4

def UMP2(mf, frozen=None, mo_coeff=None, mo_occ=None):

    from pyscf.soscf import newton_ah

    if isinstance(mf, newton_ah._CIAH_SOSCF) or not isinstance(mf, scf.uhf.UHF):
        print('hit3')
        mf = mf.to_uhf()

mo_coeff initially has only

[ 0.41972471 -0.47196477 0.3961148 0.96961923]
[ 0.28346875 0.6196045 0.67216556 -0.56718079]
[ 0.28346875 0.6196045 -0.67216556 0.56718079]] 

when the to_uhf is called it becomes the

[[[ 0.41972471 -0.47196477 -0.3961148 -0.96961923]
[ 0.41972471 -0.47196477 0.3961148 0.96961923]
[ 0.28346875 0.6196045 0.67216556 -0.56718079]
[ 0.28346875 0.6196045 -0.67216556 0.56718079]]

[[ 0.41972471 -0.47196477 -0.3961148 -0.96961923]
[ 0.41972471 -0.47196477 0.3961148 0.96961923]
[ 0.28346875 0.6196045 0.67216556 -0.56718079]
[ 0.28346875 0.6196045 -0.67216556 0.56718079]]]

but this logic has been changed in 2.5

    if mf.istype('UHF'):
        print('hit3')
        mf = mf.to_uhf()

to_uhf is called only if mf is of type UHF which doesn't make sense but this is the issue. I think the problem is with this code, I think should check for MF not being UHF and then call UHF This issue has been fixed in https://github.com/pyscf/pyscf/commit/9a152a9953f58bc632cb873bc6f9971e36216015

This code must work for the next version