dmrg inconsistency

[lucamontana]

Dear Sebastian,

I have a general problem with dmrg which is that for larger systems, i obtain different results for let say S_1 excitation energy for different number of electrons and active spaces (n,m), although n,m parameters are quite large such as (20,30) or (30,40). So it seems to be extremely difficult to obtain converged results with respect to n,m for larger simple systems (up to 100 atoms, mostly carbons and hydrogens). I have for my system, oscillations ranging from 0.5 to 4 eV for S_1 energy depending on n.m paramters. Is there any remedy for this inconsistency ?

By the way it is a very nice youtube video. Best wishes LUCA

[SebWouters]

Hi Luca,

I guess your problem as posed is really ill-defined. Which energy (DMRG-CASPT2, DMRG-SCF, DMRG) for which problems and which active spaces are you talking about?

I'm also adding @quanp because he knows a lot on active space selection.

1. When the active space is well-chosen, you should see - at least in principle - some kind of convergence of DMRG-CASPT2 energies (i.e. static plus dynamic correlation energy) for energy differences with respect to active space sizes, or so one would at least like to believe.
2. It might be that you inherently cannot expect convergence with active space size for certain kinds of problems and/or active space sizes. Remember in this light, that treating dynamic correlation with DMRG is best to be avoided. You should treat it with DMRG-CASPT2. So going for a polyene C10H12 from 10 electrons and 10 orbitals to more electrons and/or more orbitals, is something which is best to be avoided. Likewise for other problems. When you increase the active space: do you expect significant static correlation to be resolved? If yes: then the smaller active space was wrong. If no: why do you then want to increase the active space?
3. For very large active spaces, it might also be that you are stuck in local minima because of bad choices of orbital orderings, or because the bond dimension was insufficient. Did you check this?

I see this as a problem with regard to how one should use DMRG and not with bugs in the code. So I'll close it, but it remains possible to make new posts in this thread.

Sebastian

[quanp]

Hi Luca,

I agree with what Sebastian wrote. My experience with CheMPS2 is that for very large active space, you might be stuck in a local minimum, but it is rare if you use a sufficient large D. The problem you had probably come from the active space since DMRG is not a black box method, even you use a large active space. Did you check your active space in the two calculations (20,30) and (30,40)? Is it DMRG-CI or DMRG-SCF? Nevertheless, it is not surprising that you have very different total energies in the two calculations because you used different active spaces. What you should compare is relative energy, i.e. GS-S1 relative energy (the active space must be the same of course).

Best regards, Quan

[kannon92]

How can you can tell if you are stuck in a local minimum? Just by searching different solutions or is there a numerical way to say in the DMRG procedure?

[lucamontana]

Dear Kannon,

By checking energy differences with respect to addition of zero to large noise to the dmrg wave function. If both results are equal, you are NOT stuck in a local minimum.

LUCA

[lucamontana]

Dear Seb,

I mean here only dmrg-scf. I need to do it for larger active spaces to show reviewer convergence of excitation energies wrt different active spaces. I assumed that for larger active spaces choice of orbital ordering and orbital selection is less important.

Bests LUCA

[lucamontana]

Dear Quan,

It depends on what you understand under large active space, i did with chemps2 up to 84 orbitals to recover dynamical correlation a year ago. I use as initial D, D= 250. I am always comparing relative energies. As i said previously S_1 energy (First excited state).

Best wishes LUCA

[quanp]

Dear Luca,

1. Increasing noise doesn't mean that you can escape from the local minimum. You should increase D to be sure.
2. Did you do GS-S1 relative energy with the same D, same active space? DMRG-SCF is not a black box method and you should characterize your active space in all calculations.
3. Since your calculations employ very large active space (up to 84 orbitals), D=250 is too small and your results are not reliable. Moreover, if you systematically used D=250 in all calculations for both small and large active spaces, I think that it is unbalanced because the accuracy of DMRG decreases with larger active space.
4. DMRG is inefficient to describe dynamic correlation, even you use large active space. You should consider using DMRG-CASPT2, as suggested by Sebastian. Try with small active space first since DMRG-CASPT2 is very expensive.

Best regards, Quan

[lucamontana]

Dear Quan,

1. That's true.
2. I checked my results with different initial D=100,250 and 500 for (20,30) and (30,40). I get nearly but not fully converged results.
3. The case with 84 orbitals was another calculation on a small organic molecule. Due to technical problems i was unable to increase D. Actually not enough disk space.
4. I tried this, but see only a slight change of excitation energy.

Question :
Beside convergence of dmrg wrt different active spaces, i have another problem with accuracy of my relative energy wrt to experimental data. I generated the FCIDUMP with resolution of identity, as i am studying large molecules up to 300 atoms. Could this have an effect on the accuracy of dmrg results, as it is quite initial guess dependent ?

Do you recommend to start from DFT or HF orbitals ? My preference is HF, however i get sometimes better results with DFT. Best wishes LUCA

[quanp]

Hi Luca,

Again, are you sure that you have the same DMRG-SCF active space in GS and S1? Did you plot the orbitals? If you use different starting orbitals (HF vs. DFT) and you get different results, then definitely you obtain different DMRG-SCF active spaces.

Before doing DMRG-SCF, you should

• Think about your active space, which orbitals should be included? There are plenty of papers showing how to choose active space. Normally you should only include strongly correlated orbitals, i.e. orbitals with occupation number deviating from 0 and 2. The rest should be treated by PT2.
• Rotate the active orbitals to their correct places and generate FCIDUMP. If you don't you will have arbitrary initial orbitals in your active space and (for sure) bad results.
• Do a DMRG-CI first to choose a proper D.
• Do DMRG-SCF, plot the converged orbitals if they are fine.

I did not use RI for my calculation but Cholesky decomposition. I don't think RI affects your results. The fact that you have bad, arbitrary results probably comes from ill-defined, unbalanced active space.

Hope this help, Quan

[lucamontana]

Dear Quan,

Yes, i have the same DMRG-SCF active space in GS and S1.

1) my active space is quite large up to 40 orbitals, that means active space selection and orbital ordering should be much less important or not ?

2) Ok, i never rotated my orbitals, however does n't it become less important for larger active spaces ? Pi orbitals are strongly correlated, should they be put together one after another ?

3) D is fine, i checked different dimensions.

I can obtain more or less converged results wrt different active spaces, however the agreement with experimental data is not good, most probably due to my bad starting orbitals.

Which code do you use for FCIDUMP generation with Cholesky decomposition ?

Best wishes LUCA

[quanp]

Hi Luca,

1. I don't know how your molecule looks like, so I can not judge if 40 orbitals are good or not.
2. DMRG-SCF can be trapped a 'bad' orbital set if your initial guess is bad.

Sebastian and I recently implement Molcas-CheMPS2 interface and I'm using it.

Best regards, Quan

[SebWouters]

@kannon92 Regarding getting stuck in local minima:

In my opinion, the best way to test if you're in a local minimum with bond dimension D, is to go to larger virtual dimension D + D', but very approximately:

• not too tight Davidson residual tolerance, e.g. 1e-4
• only a couple of sweeps, e.g. 2
• possibly also with a small amount of noise

and then return to virtual dimension D and converge again. If the energy is the same as the energy before enlarging D to D + D', then you're most likely NOT in a local minimum.

The choice of D' depends. If D = 250, you can easily afford D + D' = 500. If D = 4000, you might want to stick with D + D' = 5000 for wall time reasons. Just don't use too small increments, like e.g. D + D' = 4050 when D = 4000. Let's say you want to increase at least 10 to 20%.

[SebWouters]

@lucamontana

1. Even when you use 40 active space orbitals, active space selection still matters. You should still verify that the GS and S1 have the same orbitals in the active space. One way you could check it is by using the GS CheMPS2_CASSCF.h5 checkpoint to start the S1 calculations from, and look at both sets of orbitals (GS and S1) and the corresponding gradient norms for rotations from GS to S1 core, active, and secondary orbitals.
2. The first part is answered above, the second part here. If the molecule has some spatial extent, and even butadiene has spatial extent, it becomes interesting to use localized orbitals and to order them according to the topology of the molecule (Fiedler vector).

@quanp : Thanks for helping!