search

openmopac / mopac

Molecular Orbital PACkage
http://openmopac.net
GNU Lesser General Public License v3.0
124 stars 32 forks source link

Expand test coverage #65

Open godotalgorithm opened 2 years ago

godotalgorithm commented 2 years ago

Testing of MOPAC is based on simple regression tests that compare MOPAC output files between reference output files and the output files presently being generated by MOPAC. The comparisons are implemented in Python (tests/compare_output.py), orchestrated by CTest (tests/CMakeLists.txt), and monitored for coverage by CodeCov, as indicated by the coverage badge on the repository's main README file. Right now, the coverage is relatively low (68% at the time of writing this), and it should probably be in the high 80's or low 90's.

The submission of new tests that expand testing coverage is very welcome, particularly with an emphasis on any common use cases that are not presently represented in the existing tests. The tests in the main tests directory are a combination of historical regression tests used by Jimmy Stewart and validation tests of the INDO feature used by Rebecca Gieseking, and the keywords subdirectory contains keyword-specific tests I've written in the process of migrating keyword documentation between MOPAC's old website and the new website that is still in development.

Testing is particularly important for MOPAC because of its dual role as a popular, widely-used, and venerable electronic structure package and as the long-time primary codebase for the computational chemistry research work of Jimmy Stewart. In its first role, the core functionality of MOPAC has been heavily tested in practice by its large user base. However, there are many features and options in MOPAC produced over the course of Dr. Stewart's scientific career and many conceptually valid combinations of these features and options, not all of which see regular use. Some of these features may not be as robust as the core functionality, and if problems are encountered, it will be either by developers during the process of expanding test coverage or by users who are trying to apply MOPAC in uncommon and novel ways. The former scenario is much preferred to the latter.

flatstik commented 2 years ago

I will happily submit everything I test, but at the same time, I am hoping to get my thesis done which is related to Q's & A's here :)

zfekete2 commented 1 year ago

Here I attached a handful of tests, which are interesting to me (and cover some missing A-F keywords). @godotalgorithm let me know if these kind of files are useful.

extra-tests.zip

miroi commented 1 month ago

Hello,

in addition to this issue, I just installed Mopac Linux package mopac-22.1.1-linux.run, but I do not see the test directory in the installation:

miroi@MIRO:/opt/mopac/.ls
CITATION.cff  COPYING.lesser       Licenses/  components.xml  installerResources/  mopac.ico    uninstall-mopac*     uninstall-mopac.ini
COPYING       InstallationLog.txt  bin/       installer.dat   lib/
godotalgorithm commented 1 month ago

MOPAC's tests are mainly for development purposes, so they are not included in the binary distribution.

I have been gathering statistics on how people commonly use MOPAC, and I might eventually add a set of examples to the distribution that reflect standard use cases, but I don't think it makes sense to distribute the full set of tests, especially if testing coverage increases and the number of tests grows a lot.

miroi commented 1 month ago

Would you please share with me this extra set of MOPAC examples. Thanks.

godotalgorithm commented 1 month ago

My statement about adding a new set of examples was about future plans. I don't have a set that is readily available.

All of the tests that are readily available and actively used in development are in the tests directory in MOPAC's GitHub repository. In particular, the port.mop file is the main set of tests that has been compiled over MOPAC's decades of development.