Full and privileged xtb integration

[matterhorn103]

Hi ghutchis,

Saw that you did significant work on adding an interface between Avogadro and various force field implementations, including the Python API for XTB. Currently this seems to use just the GFN-FF force field method, in alignment with the other options for optimization. This is fine, but since optimizations with the normal semi-empirical GFN2-xTB are also extremely fast, and the accuracy improvement over force fields is significant, it would be cool to include that. Likewise, it would be cool if the user could run CREST calculations directly from Avogadro, to find low energy conformers prior to doing DFT, in the same way various people I know use SPARTAN.

At the moment you seem to be using the xtb-python API. This is apparently then in turn limited to what is available in the xtb C API. Which I guess mean CREST and things is right out.

Also currently the user has to install python, and install xtb-python using conda, and set up their python environment, which is not something 95% of Avogadro users (lots of students) can even begin with, at least not without their hands being held throughout the process.

I have a suggestion.

Since xtb is completely open source and freely available, would it not be possible to simply package it with Avogadro? And then, on request by the user, run calculations using system calls? As in, complete native integration, as the privileged tool for optimization and calculation?

The license for xtb is LGPL/GPL, so if it was packaged as a dynamically-linked binary it should be ok with the Avogadro license, right? The binaries come to 87 MiB on my PC, which I guess is significant, but it can do a lot!

I suggest this because I find xtb to be a truly excellent tool for general use (as it seems you do too based on e.g. this thread), and I really think it would be great for chemical education and open-source chemistry if the use of it was widely accessible via a GUI, especially one as widely distributed as Avo.

Alternatively, I have some intermediate-level Python, including a little PyQt/PySide, but no experience with C++. Do you reckon making an optional add-on or plugin that can be downloaded and set up with one click, that included the xtb binary, would be a) technically feasible and b) within my capabilities to contribute?

Of course it would also be an option to have an interface to xtb go via the interface you built in for force fields in 1.98, but then it would have to not refer exclusively to force fields all the time!

I think effectively making Avogadro a portal to xtb for non-computational chemists would really give a sort of synergistic benefit to both tools.

[avo-bot]

This issue has been mentioned on Avogadro Discussion. There might be relevant details there:

https://discuss.avogadro.cc/t/full-and-privileged-xtb-integration/4953/1

[ghutchis]

For anyone following this, the forum has the on-going discussion.

[ghutchis]

1. [ ] Ensure we can get URLs for xtb binaries (needed for both plugin option or binary installs):
   • [x] Linux x64 (for AppImage / Flatpak)
   • [x] Win x64
   • [ ] Mac x86
   • [ ] Mac arm64
2. [ ] Make list of needed features
   • [ ] Optimize geometry via xtb --opt
   • [ ] Optimize unit cells via xtb --opt
   • [ ] Generate molden file for orbitals / electron density
   • [ ] Generate vibrational spectra for current geometry
   • [ ] Other features?
3. [ ] Start implementation for testing via Python command scripts

[avo-bot]

This issue has been mentioned on Avogadro Discussion. There might be relevant details there:

https://discuss.avogadro.cc/t/full-and-privileged-xtb-integration/4953/14

[matterhorn103]

The xtb repo readme says:

Statically linked binaries (Intel Compiler) can be found at the latest release page, a version for Linux (Intel 18.0.2, GLIBC 2.19) and Windows (Intel 2022) is provided. The xtb program and library are packaged on conda-forge for Linux (x86_64, aarch64, ppc64le) and MacOS (x86_64, arm64). For homebrew users a custom tap is available at grimme-lab/homebrew-qc providing prebuilt MacOS/x86_64 binaries, for MacOS/arm64 binaries will be compiled on installation automatically.

crest is also available from the linked repo. I'm not really sure how homebrew works, as I only use Linux and Windows these days, but it seems like this is a way we could get the macOS binaries.

[ghutchis]

Yeah, I've been looking at the Mac binaries. At the moment, xtb at least is compiled in a way that requires other libraries from homebrew. (It's a bit like conda - it's a package manager.)

It's certainly possible to compile something close to a static xtb binary on Mac as well as the associated libxtb. I would guess crest should be fairly easy too.

One question is that the static binaries are available for Linux and Windows, but not libxtb for using the API directly.

On the other hand, for a first task of "make some Python commands with the binaries" we should be in good shape.

[avo-bot]

This issue has been mentioned on Avogadro Discussion. There might be relevant details there:

https://discuss.avogadro.cc/t/managing-python/5098/1

[avo-bot]

This issue has been mentioned on Avogadro Discussion. There might be relevant details there:

https://discuss.avogadro.cc/t/full-and-privileged-xtb-integration/4953/56

[matterhorn103]

We can maybe think about closing this issue, right? I think avo_xtb provides a good proof of principle, as it offers the key basic xtb functionality, and as discussed we'll both be working on expanding the range of CREST methods available. So that's the "full" integration.

Originally I approached the idea with the aim of integrating it directly within Avogadro (i.e. having the plugin pre-installed, bundling the xtb binary) and gently pushing users towards it – "privileged" integration. @ghutchis – your opinion on that seemed to vary, are you keen to explore that possibility further or are you pretty happy with the status quo?

If the latter, we can close this issue :)

[ghutchis]

Let's close this for now. For one, it seems as if the Grimme group is switching towards tblite.

I think since we can install conda pre-requisites, the main thing would be to get more users to install conda-forge.