Parameter handling incorrect for orbital rotation with RHF wavefunction with non-zero total spin

[markdewing]

In the case with a restricted Hartree-Fock (RHF) wavefunction, the up and down spins have the same set of MO coefficients (and the same rotation matrix parameters). If the total spin is non-zero, there are different numbers of electrons for each spin group. This leads to a differing number of rotation parameters (well, a different number of rotation parameters with non-zero derivatives)

Two copies for RotatedSPOs get created, and the two calls to buildOptVariables have the same number of MOs, but a different number of electrons, leading to a different number of rotation parameters. The aggregation of OptimizableObjects keys off the name of the class (in UniqueOptObjRefs::push_back). There is a single name for both RotatedSPOs classes, and only one gets included when gathering the variational parameters in TrialWaveFunction::checkInVariables. This leads to some of the parameters getting missed and the derivatives of those rotation parameters being incorrectly set to zero.

Take the ground state of a carbon atom for a concrete example. It has 4 electrons in one spin group and 2 electrons in the other spin group. The number of MOs is 9, the number of parameters in the up group is 20 and the down group is 14. There is considerable overlap in the parameters, and the total number of unique parameters should be 24.

   Single Slater determinant
   -------------------------

     Determinant
     -----------
      Name: rot-spo-up   Spin group: 0   SPO name: rot-spo-up

      Setting delay_rank to default value 1
      Using rank-1 Sherman-Morrison Fahy update (SM1)
      Running on CPU.
Orbital rotation using global rotation
nparams_active: 20 params2.size(): 0
                Parameter name               Value
rot-spo-up_orb_rot_0000_0004                 0.000000e+00 0 1  ON 0
rot-spo-up_orb_rot_0000_0005                 0.000000e+00 0 1  ON 1
rot-spo-up_orb_rot_0000_0006                 0.000000e+00 0 1  ON 2
rot-spo-up_orb_rot_0000_0007                 0.000000e+00 0 1  ON 3
rot-spo-up_orb_rot_0000_0008                 0.000000e+00 0 1  ON 4
rot-spo-up_orb_rot_0001_0004                 0.000000e+00 0 1  ON 5
rot-spo-up_orb_rot_0001_0005                 0.000000e+00 0 1  ON 6
rot-spo-up_orb_rot_0001_0006                 0.000000e+00 0 1  ON 7
rot-spo-up_orb_rot_0001_0007                 0.000000e+00 0 1  ON 8
rot-spo-up_orb_rot_0001_0008                 0.000000e+00 0 1  ON 9
rot-spo-up_orb_rot_0002_0004                 0.000000e+00 0 1  ON 10
rot-spo-up_orb_rot_0002_0005                 0.000000e+00 0 1  ON 11
rot-spo-up_orb_rot_0002_0006                 0.000000e+00 0 1  ON 12
rot-spo-up_orb_rot_0002_0007                 0.000000e+00 0 1  ON 13
rot-spo-up_orb_rot_0002_0008                 0.000000e+00 0 1  ON 14
rot-spo-up_orb_rot_0003_0004                 0.000000e+00 0 1  ON 15
rot-spo-up_orb_rot_0003_0005                 0.000000e+00 0 1  ON 16
rot-spo-up_orb_rot_0003_0006                 0.000000e+00 0 1  ON 17
rot-spo-up_orb_rot_0003_0007                 0.000000e+00 0 1  ON 18
rot-spo-up_orb_rot_0003_0008                 0.000000e+00 0 1  ON 19

     Determinant
     -----------
      Name: rot-spo-up   Spin group: 1   SPO name: rot-spo-up

      Setting delay_rank to default value 1
      Using rank-1 Sherman-Morrison Fahy update (SM1)
      Running on CPU.
Orbital rotation using global rotation
nparams_active: 14 params2.size(): 0
                Parameter name               Value
rot-spo-up_orb_rot_0000_0002                 0.000000e+00 0 1  ON 0
rot-spo-up_orb_rot_0000_0003                 0.000000e+00 0 1  ON 1
rot-spo-up_orb_rot_0000_0004                 0.000000e+00 0 1  ON 2
rot-spo-up_orb_rot_0000_0005                 0.000000e+00 0 1  ON 3
rot-spo-up_orb_rot_0000_0006                 0.000000e+00 0 1  ON 4
rot-spo-up_orb_rot_0000_0007                 0.000000e+00 0 1  ON 5
rot-spo-up_orb_rot_0000_0008                 0.000000e+00 0 1  ON 6
rot-spo-up_orb_rot_0001_0002                 0.000000e+00 0 1  ON 7
rot-spo-up_orb_rot_0001_0003                 0.000000e+00 0 1  ON 8
rot-spo-up_orb_rot_0001_0004                 0.000000e+00 0 1  ON 9
rot-spo-up_orb_rot_0001_0005                 0.000000e+00 0 1  ON 10
rot-spo-up_orb_rot_0001_0006                 0.000000e+00 0 1  ON 11
rot-spo-up_orb_rot_0001_0007                 0.000000e+00 0 1  ON 12
rot-spo-up_orb_rot_0001_0008                 0.000000e+00 0 1  ON 13

After calling checkInVariables, only one spin's rotation parameters get included:

 TrialWaveFunction "psi0" has 1 optimizable objects:
   'rot-spo-up' optimized
 Variational subset selects 20 parameters.

One simple solution (hack?) would be to add the number of rotation parameters to the name of the class returned from getName.

[markdewing]

I tried the hack solution of adding the number of rotation parameters to the name of the class. It did not work.

The aggregation of OptimizableObjects by name worked, and all the rotation parameters were present in the list of parameters. The problem comes from the parameters shared between the two instances of RotatedSPOs.

There are two instances of RotatedSPOs created - one for each spin state, and one instance of the LCAO class (most importantly, the MO coefficient matrix). Each instance of RotatedSPOs holds a different, conflicting view of the rotation matrix.

I tried to ensure each RotatedSPOs instance had a non-overlapping subset of rotation parameters. That did not work, either. Even with the history method of accumulating rotations.

I think what will need to happen is the rotation matrix information (previous set of parameters to compute the delta rotation, full rotation parameters for global method) will need to be shared between the RotatedSPOs instances.