The developer wish list

dgasmith commented 8 years ago

This is a list of things that are generally agreed as "good things to do", but nobody has actually tackled yet. The hope is that this is used a central repository of "have you thought about this" comments. Note that this is a not a feature wish list.

High

[ ] Deprecate PSIO by moving to HDF5. With a temporary under the hood PSIO move to HDF5.
[x] Remove boost::python and replace with PyBind11.
[x] Possibly remove Boost with C++11 features. Std managed pointers are high on the list.
[x] Switch Psi4 from a binary to a library
- [x] Allow more flexible external libraries
[x] Remove C-side python calls. DFT-D3, DFT functionals, BasisSet parsing, etc.
[x] Add a dictionary to the Wavefunction object thats holds the current Psi Variables.
Medium
[ ] Rewrite LibOptions as a property map or just a general dictionary. Needs to be more object oriented, less global, and capable of encompassing all QCDB.
[x] Full Python3 support. This is mostly done, but small sections are not Python3 compatible.
[ ] Ability to combine DF fitting sets via partial decomposition of a expanded Coulomb metric.
[x] ECP would be a great addition.
[ ] Purge all char* from Psi4. Issues with dropping pointers and python exportation.
[ ] A smarter SCF solver that can change iterations based on current conditions, see #211.
Low
[ ] More flexibility in compilation of integrals. For example compiling 3-index and Coulomb metric integrals at a higher AM than the conventional basis set.
[x] 4th CMake rewrite. Currently overly cautious about what needs to be recompiled. Our CMake files should ensure that the compilers support all the features that we need.
[ ] EFP Gradients: https://github.com/psi4/psi4private/issues/70
[ ] Uniform space setter for DMRG/CI/MCSCF/etc computations.
[ ] change_file_namespace should be able to tie multiple files together and should return the status rather than None, #645.
[x] Molecule parser can accept atomic numbers instead of symbols, #418. (fixed by #953)
[ ] DETCI for more than 256 orbitals
[ ] Automatic choice between rhf/uhf/rohf/cuhf for input molecules.
[ ] Potenital integral derivative performance can be improved, see #3.
[ ] Allow a DECON keyword to basis sets #43.
[ ] Remove char * for std::string.

Modify, expand, delete as desired. If you take up a feature make sure to post here so that we do not duplicate effort.

jgonthier commented 8 years ago

Medium

[ ] Parallelize the mints integral generation: Needs to use asynchronous I/O to have one process writing to disk while others compute. Basic technology to do that is in the new PKJK object.
[ ] Reduce PKJK I/O Preliminary experiment shows that storing only the ordered supermatrix for J and reordering it on the fly when forming K is faster than I/O for storing/reading ordered supermatrix for K.
[ ] Sieve PKJK Currently PKJK only uses sieving for integral computations. Could use it to only I/O significant integrals. We would probably need a specialized iterator that goes only through significant integrals and gives us back the absolute indices for building J and K.

jgonthier commented 8 years ago

Medium

[ ] Design and implement a memory manager. It should keep track of the memory allocated and provide the user with the currently available memory.

loriab commented 8 years ago

Purportedly, @bennybp has a memory manager in pulsar.

loriab commented 8 years ago

Moved from #476 so as to not clutter issues.

[ ] Here lie uses for general Cholesky machinery.
- [ ] Fock-type terms in DF-SAPT, particularly higher-order, see #317
- [ ] If you care about 6-zeta accuracy, you probably oughtn't to be using density fitting. Nevertheless, having orbital basis sets for which no auxiliary is appropriate (even as overkill, like QZ for sto-3g) is troubling for the auxiliary defaulting machinery. On-the-fly construction could help. Maybe a more plausible case is when auxiliary basis sets aren't available for a particular element or for tight functions.

Moreover, Susi Lehtola comments:

Well, you could do compact atomic Cholesky to construct a fitting basis for an arbitrary orbital basis set on-the-fly. [doi 10.1063/1.3116784]

loriab commented 7 years ago

@CDSherrill requests a disk use estimate. Could just be final disk use as a system call before cleaning (https://github.com/psi4/psi4/blob/master/psi4/run_psi4.py#L196).

CDSherrill commented 7 years ago

Yes, this would be very helpful so that users will know how much disk is needed (even approximately) for subsequent jobs of this type. It is also helpful to people planning to buy new hardware and who need to know how much disk to buy.... one can look back through old outputs on some of the challenging computations and see how much disk they used.

On Wed, Nov 16, 2016 at 3:06 PM, Lori A. Burns notifications@github.com wrote:

@CDSherrill https://github.com/CDSherrill requests a disk use estimate. Could just be final disk use as a system call before cleaning ( https://github.com/psi4/psi4/blob/master/psi4/run_psi4.py#L196).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/psi4/psi4/issues/402#issuecomment-261056552, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9QdgSg1IZHU-ezlwNRXjWKX3oEMSQaks5q-2JfgaJpZM4IvHrd .

susilehtola commented 7 years ago

detci memory size checks would be nice. Signed, someone who has crashed his computer a few times by trying to do a too big calculation.

loriab commented 7 years ago

[ ] Implement logging for PsiAPI mode so that output files (currently w/o Psi4 header or any record of input commands or options) aren't so mysterious. At minimum, print to output the psi4.header() at file creation and e.g. >>> aggh = psi4.energy("scf") and >>> psi4.set_options({"scf_type": "pk"}) to track what was done.

loriab commented 7 years ago

[ ] API-like loading of basis sets. If one has the coef and the exp, shouldn't have to write them out to a .gbs, should be a constructor to take them directly.

loriab commented 7 years ago

[ ] Adapt CC* modules to take in Wfn so that truncated virtual space proceeds transparently. Feature request (http://forum.psicode.org/t/rvs-approximations-for-cc2-or-adc-2/504)

loriab commented 7 years ago

[ ] Matt Schieber has info on KMP affinity and thread binding schemes that hurt and help performance. See if the one that helps can be encoded at runtime (knowing some /proc/cpuinfo) or otherwise advise in docs.

susilehtola commented 7 years ago

[ ] CISD module should be able to compute density matrices, so that the NOs could be used as initial guesses for detci

loriab commented 7 years ago

[ ]

sp = single_point('scf')
sp.compute() # ?
wfn = sp.wfn()
energy = sp.energy()

susilehtola commented 7 years ago

[ ] Implement O2 method, i.e. orbital-optimized scaled opposite-spin MP2 [R. C. Lochan and M. Head-Gordon, J. Chem. Phys. 126, 164101 (2007)]. (Looks like there is a scaled opposite-spin MP2 method in dfocc?)

susilehtola commented 7 years ago

[ ] Implement exchange guess for virtual orbitals for CAS calculations. (Improved virtual orbitals by rediagonalizing virtual space with respect to exchange operator.)

loriab commented 7 years ago

[ ] Unify origin-setting for properties/integrals. Properties by oeprop get their origin from options (https://github.com/psi4/psi4/blob/master/psi4/src/psi4/libmints/oeprop.cc#L775-L808). Mints integrals get their origin from default 0,0,0 or from options. But with Mints, you sometimes want to set the origin by argument (default 0,0,0) in API. Unification so that useful from both input file and API would be nice. Also maybe an easy center-of-charge spec.

loriab commented 7 years ago

[x] Can libmints get away with not having MAX_AM_ERI passed to it? That way one could switch out libint.sos on the fly (conda can do this). Would have to find a way to still exit gracefully if insufficient AM and give decent error messages. EDIT: Can't be done, as the header sets data structure sizes as well. Settling on AM=8 for conda linux instead.

loriab commented 7 years ago

[ ] Examine properties function in driver (that encodes the extended workflows beyond sp or time-trivial keyword needed to compute CC properties) and the oeprop syntax that most all other methods use. Devise and implement a unified interface that still works with the code.

robertodr commented 7 years ago

[x] Add FCIDUMP capabilities within Psi4. There are currently many third-party plugins that do that and it seems needed enough to have it within the main code.

Done in #872

loriab commented 7 years ago

Beginner-Friendly (after py-side SCF)

Renovate SCF damping capabilities

[ ] right now we're only damping D. There are rumors that should also damp F or there's an iteration mismatch between D & F. Make sure damping is working right (presently only test case is scf-coverage and damping severely hurts convergence
[ ] right now we damp to the very end of iterations. Maybe damp only to 1e-4 (after which soscf might be better choice, now that DGAS has implemented) controlled by DAMPING_CONVERGENCE.
[ ] here's a paper with potentially useful schemes https://arxiv.org/pdf/1302.6099.pdf
[ ] consider (this may be separate project) more advanced driver logic. perhaps something along the lines of (DGAS) "Damp first iteration, then DIIS, if density is stuck take stability step, if grad < 1.e-4 and target < 1.e-8 take SOSCF, else DIIS. if density oscillation (dRMS/oscilation) > 1.e2 add damping"

susilehtola commented 7 years ago

Related to @loriab 's previous comment... damping F is a lot more better than damping D, since instead of intrapolating inside a fixed Hilbert space of density, you're actually looking at different densities. You converge faster if you damp F than if you just damp D, and you're less likely to land on a saddle point.

loriab commented 6 years ago

Make use of doublet/triplet c-side. In particular, https://github.com/psi4/psi4/pull/737/files#r121282237

loriab commented 6 years ago

things we can do after dumping py27 (just to keep track)

[ ] delete all __future__ statements
[ ] simplify molutil and qcdb/molecule.py raw_* functions

loriab commented 6 years ago

[from @dgasmith psi4/psi4#928 ] File names are going haywire a bit with different conventions for either PSIO or "user facing names" like Molden and cube files. Anyone have a good idea how to rectify this? It would also be good to have restart work for gradients, Hessians, and properties as well. Ill add this to the wish ilst.

j3mdamas commented 6 years ago

As per discussed with @loriab in https://github.com/psi4/psi4/issues/933, it would be interesting to have a Windows native compilation for psi4

loriab commented 6 years ago

Possible easy-start issues

[ ] rVV10, comments starting here
[ ] Wigner tunneling correction, comments
[ ] Grimme low-freq vib correction

robertodr commented 6 years ago

[ ] PCM gradients à la ECP, whereby the in vacuo terms are computed analytically and the PCM terms numerically. This can actually be done already on top of #953.
[ ] PCM analytic gradients. This is more of a PCMSolver-side problem and a time investment I can't currently make. For those interested, there is this (very broken) PCMSolver/pcmsolver#140 which should make much easier implementing the cavity terms for the gradient and Hessian.

loriab commented 6 years ago

Plugin wish list:

[ ] allow compiled pluginname.so to have a name other than matching import pluginname, so that python import system can't mistake the .so for the actual module. (see https://github.com/edeprince3/gpu_dfcc/issues/2#issuecomment-397649361)

loriab commented 6 years ago

From @dgasmith, relevant to the smart SCF driver

[ ] We should look into a SCF mode that saves some grid components, they are not that big and the savings can add up for high iterations.

loriab commented 6 years ago

So we don't forget it

[ ] https://github.com/psi4/psi4/pull/1075#discussion_r203034758

JonathonMisiewicz commented 6 years ago

Have findif intercept and rerun suspect SCFs. Definitely wait for this until after findif is accepted, findif is converted from list based to dict based (so we can store SCF at the displacement as metadata) and we have some preliminary smart SCF.

loriab commented 6 years ago

[ ] PSIO_REMOVE_FILE/ PSIO_KEEP_FILE? from #1310
[ ] matrix copy issue https://github.com/psi4/psi4/pull/1267/files#r228203247

loriab commented 6 years ago

[ ] -D(abc) work needs to continue, see https://github.com/psi4/psi4/pull/1151#issuecomment-417101476 and below
[ ] add named per-parameter access to DFT functionals now that we can get names from libxc

JonathonMisiewicz commented 5 years ago

For findif computations, have the preliminary displacements needed to compute the gradient be funneled into the frequency displacements.

loriab commented 5 years ago

[ ] beta orbitals in mo extents
[ ] mo polarity https://github.com/psi4/psi4archive/commit/d337912cc48fe14aa8ae78820581fc8f0495383a

susilehtola commented 5 years ago

possibility to override charge states of individual atoms in SAD guess

loriab commented 5 years ago

been adding some things that are correct but not full-stack hooked up. here are the reminders to do so:

[ ] after #1491, add the ability to call ATM from functional., e.g. energy('b3lyp-d3-atm)`
[ ] after #1491, MP2D gradients still need addressing
[ ] after #1490, set up a energy(sapt0inf) call (if component anaylysis deems suitable) and add it to the qcvar parade. Also, figure out why it's only in fisapt and is called through dft-sapt.

loriab commented 5 years ago

save one-electron ints in wfn so that (1) not writing to disk, thus freeing SCF from disk and thread-safety issues, and (2) better carry around external potentials btwn modules.

loriab commented 5 years ago

[ ] dft functionals in https://github.com/psi4/psi4/blob/master/psi4/driver/procrouting/dft/libxc_functionals.py pull all their citation metadata from libxc, thus avoiding duplicated work. Functionals that need separate X & C from libxc currently have metadata redefined in psi. Change this so that where X & C pulled separately from libxc have the same citation, this gets adopted as total citation. discussed a bit in https://github.com/psi4/psi4/pull/1597

loriab commented 5 years ago

[ ] Investigate Susi comment on possible advantages for ints screening if we switched the arithmetic for omega functionals.

susilehtola commented 5 years ago

[ ] Implement potential-based small-basis guess as described in Chem Phys Lett 531, 229 (2012). The method avoids doing projections altogether, but the main problem is that one needs cross-basis Coulomb and exchange integrals (IJ|kl) and (Ik|Jl) where IJ are in the new, large basis and kl are in the small input basis. For DFT, the density variables would be evaluated in the small basis, and the quadrature would be done in the new basis.

loriab commented 4 years ago

[ ] MemJK disk algorithm therein could have omega thrown in. background in https://github.com/psi4/psi4/pull/1749

susilehtola commented 4 years ago

[x] seminumerical exchange engine. doi:10.1021/acs.jctc.9b00860 shows a 10x speedup over LinK (NOTE: could be considered fulfilled via the addition of COSX to Psi4 in PR https://github.com/psi4/psi4/pull/2567, although the paper linked seems to describe a method other than COSX specifically.)

hokru commented 4 years ago

[ ] direct DF-J/K integral generation for better parallel scaling (NOTE: Direct DF-J has been added to Psi4 in conjunction with LinK and COSX as of PRs https://github.com/psi4/psi4/pull/2762 and https://github.com/psi4/psi4/pull/2567, respectively.)

loriab commented 3 years ago

[ ] @jturney found https://github.com/pybind/pybind11_json to convert python dict to nlohmann's json library. could be useful for qcsk <-> psi4.core.BasisSet roundtripping and options conversion

TiborGY commented 2 years ago

[ ] Replace all __FILE__, __LINE__ and __PRETTY_FUNCTION__ macros with std::source_location when Psi4 transitions to C++20 minimum requirement

TiborGY commented 2 years ago

[ ] Beyond just debug/release CMake build type, have a "profiling" build where both debugging and optimization are enabled. This is required if one wants to run perf/Vtune and get meaningful results. With modern compilers the debugging symbols contain enough information that compiler optimizations do not make debugging/profiling impossible.
[ ] Investigate the possibility of building everything with LTO. Possible benefits: smaller binary sizes, performance improvement, unused functions are automatically stripped.

hokru commented 2 years ago

Beyond just debug/release CMake build type, have a "profiling" build where both debugging and optimization are enabled. This is required if one wants to run perf/Vtune and get meaningful results. With modern compilers the debugging symbols contain enough information that compiler optimizations do not make debugging/profiling impossible

Before making a custom option, perhaps RelWithDebInfo is enough?

loriab commented 2 years ago

CD gradient tech. @JonathonMisiewicz located some papers: https://aip.scitation.org/doi/10.1063/5.0087261 and https://onlinelibrary.wiley.com/doi/10.1002/qua.24563

davpoolechem commented 1 year ago

Some things to add to the wishlist from my end:

[ ] Update COSX to support arbitrary grids via modification of X-matrix handling (see https://github.com/psi4/psi4/issues/2890 for further details).
- Susi notes: I don’t think one needs a separate formulation, just propagating the sign to the A tensor would suffice since it carries a grid index
- Further update: this wish is now addressed in https://github.com/psi4/psi4/pull/2931.
[ ] Add CFMM and/or DF-CFMM to Psi4 officially. These already exist in Psi4 development branches, but need to be updated to match modern Psi4 developments (e.g., current formulations of CompositeJK) and actually put into Psi4. The first PR for these methods already exists (https://github.com/psi4/psi4/pull/2665).
[ ] Support for J-Engine. J-Engine is a nice algorithm by the Head-Gordon group to help reduce the prefactor of J-build calculations via reformulation of how integral intermediates are formed and contracted with the density matrix to form Coulomb matrix elements. As it deals with shuffling around how integral intermediates are formed and combined with the density, any J-Engine effort will likely require coordination with the Valeev group and development within Libint2 itself (although the Valeev group did seem interested in J-Engine from what discussions I had with their researchers).
[ ] Support for mixed basis sets in TwoBodyAOInt sieving.

psi4 / psi4