IAO fragmentation for DMET

[maxnus]

21 reverted the removal of the IAO fragmentation for DMET.

I'm fine with leaving it available for DMET (but leaving the documentation as it is, saying it's not available?)

One of the reasons was that the change from IAO to SAO breaks the miniumum basis (STO-6G) DMET tests; this only happens because the IAO reference basis is taken to be the 'minao' basis of G. Knizia, but I think it is hard to justify that the IAOs for a minimal basis set should be anything different than the unchanged basis set itself. I will update the default basis at least for the STO-nG family, while leaving the tests untouched.

I actually think we should protect the user from using an insufficient fragmentation in a more general sense (which would also catch if some atoms were forgotten for example). That is, we could add two routines, fragmentation_complete() and fragmentation_occupied_complete(), which check that the entire occupied+virtual and occupied space is spanned by orthonormal fragment orbitals, respectively. This can be tested either on __exit__ of the fragmentation context manager, or at the beggining of kernel() and print a warning if it returns False.

[maxnus]

On closer look, changing IAO to SAO fragmentation doesn't seem to break the (E)DMET tests for me at all - @cjcscott is it only the skipped ebcc tests that break?

[ghb24]

I like that idea of checking the 'completeness' of the fragmentation.

I don't really understand the fundamental problem in having a non-spanning fragmentation of the virtual space in (E)DMET though. Of course, you can not hope to expand to 'in-method' completeness, but that is still a physically valid method?

[maxnus]

I don't really understand the fundamental problem in having a non-spanning fragmentation of the virtual space in (E)DMET though. Of course, you can not hope to expand to 'in-method' completeness, but that is still a physically valid method?

Depends what you do with it - the democratic DMs do not trace correctly, so calculating any properties from them (including the total energy) does not really make sense. If someone is only interested in "cluster properties" this wouldn't apply of course.

One way to combine both is just to check the completeness when calling any routine that assumes a complete fragmentation, such as make_rdm1 and print a warning

[ghb24]

Sure - I like that idea.

[maxnus]

I added a few such check here There might be more functions where we want to add the check

[cjcscott]

It may only be the ebcc tests skipped when you haven't got it installed that ended up failing- I guess we've removed all non-minimal basis DMET tests and only have minimal basis EBFCI tests. Even without that the point still stands. Few other qs:

• If you're using a chemical potential optimisation in DMET surely the democratic RDMs will trace properly and not be guaranteed to give a terrible result (for that reason at least)?
• This all seems to assume that a user would never want to treat part of the system at the mean-field level (which is all a non-spanning fragmentation really means). While this often makes sense, if you're looking at, for instance, a local reconfiguration it might not be practical or efficient to have a spanning fragmentation of the entire system. Adding a flag to ee7f9d2861eff9bdf5b8e05dbdda78ba7dae93a9 to avoid anyone getting absolutely swamped with warnings if they were to run such a calculation might be a nice.

[cjcscott]

aka, adding an option to allow someone to say I don't want to be warned about this on this run

[maxnus]

I guess we've removed all non-minimal basis DMET tests and only have minimal basis EBFCI tests. Even without that the point still stands.

I will change the non EBFCI tests to SAO, since they pass for me and leave the EBFCI tests at IAO OK?

If you're using a chemical potential optimisation in DMET surely the democratic RDMs will trace properly and not be guaranteed to give a terrible result (for that reason at least)?

If I recall correctly, trying to optimize the chemical potential with a non-complete fragmentation almost never converges

This all seems to assume that a user would never want to treat part of the system at the mean-field level (which is all a non-spanning fragmentation really means). While this often makes sense, if you're looking at, for instance, a local reconfiguration it might not be practical or efficient to have a spanning fragmentation of the entire system. Adding a flag to https://github.com/BoothGroup/Vayesta/commit/ee7f9d2861eff9bdf5b8e05dbdda78ba7dae93a9 to avoid anyone getting absolutely swamped with warnings if they were to run such a calculation might be a nice.

This is a good point, however I would suggest that the user has to add the "HF fragment" manually in this case, i.e. something like

with emb.sao_fragmentation() as frag:
   frag.add_atomic_fragment(0)
   frag.add_atomic_fragment([1,2,3], solver='HF')   

which would then not print a warning

[ghb24]

The other way you can do it (which is essentially what 'ab initio' DMFT does), is optimize the bath chemical potential for each cluster solver, s.t. the number of electrons in each fragment is the same as HF in the fragment (rather than a global chemical potential). This then does not require a fully spanning fragmentation, but rather just an occupied fragmentation to ensure electron number is conserved. Correlation essentially changes the fragment state, but does not net change the number of electrons between fragment and bath in each cluster compared to MF. Electron number is then conserved in the democratically partitioned DMs (and potentially adjusted in a 'global' self-consistency which changes the MF state). This could then legitimately use an IAO fragmentation (albeit without any 'dynamic' correlation)?

[maxnus]

So you are saying fragment electrons are only allowed to rearrange within the IAO space - the "PAO" space can only be occupied by bath electrons? I don't really see the point of this as it's a fairly uncontrolled approximation and doesn't become exact in the full bath limit, only in the full fragment limit. But I guess it is possible to do such a calculation with the current code and it shouldn't trigger a warning

[cjcscott]

If I recall correctly, trying to optimize the chemical potential with a non-complete fragmentation almost never converges

I don't see why this should necessarily be the case- self-consistency could be interesting though, would need to relax the traceless condition on the correlation potential...

This is a good point, however I would suggest that the user has to add the "HF fragment" manually in this case, i.e. something like

with emb.sao_fragmentation() as frag: frag.add_atomic_fragment(0) frag.add_atomic_fragment([1,2,3], solver='HF')

which would then not print a warning

Depending on what you're doing, defining the space treated at HF level might well be non-trivial. Should also note that in EDMET this will be effectively treated at the level of RPA via other mechanisms, so could for instance include higher virtual orbitals outside the IAOs or similar.

So you are saying fragment electrons are only allowed to rearrange within the IAO space - the "PAO" space can only be occupied by bath electrons?

I suppose this just assumes that the mean-field electron distribution at the mean-field level is correct; probably more reasonable in DMFT using a DFT reference, but might not be terrible. And if you're just trying to capture physics relevant to a particular problem, rather than the whole system, this might give a balanced representation for energy changes over some transformation.

But I guess it is possible to do such a calculation with the current code and it shouldn't trigger a warning

Wouldn't this trigger a warning on calculating the energy from the democratic RDMs?

[maxnus]

I don't see why this should necessarily be the case- self-consistency could be interesting though, would need to relax the traceless condition on the correlation potential...

Yes, this was from experience only

Depending on what you're doing, defining the space treated at HF level might well be non-trivial.

We could add a add_remaining_fragment() to the fragmentation class, which just add a fragment will all remaining orbitals? I am not familiar with the EDMET requirements, but I think if edmet.make_rdm1 does not require a complete fragmentation, you can simply shadow this routine?

Wouldn't this trigger a warning on calculating the energy from the democratic RDMs?

Yes true, unless the remaining virtual space is added as a HF fragment. I guess another option is to always add a HF remainder fragment automatically, if the fragmentation is not complete