xTB conda installation not registered correctly with QCEngine

[xiki-tempula]

Describe the bug I would like the QCEngine to find the xTB installed with conda.

To Reproduce Create a new environment and install the xtb as well as the QCEngine

conda create -n torsiondrive -c conda-forge rdkit qcengine torsiondrive xtb

Then I tried to use xTB to run the torsiondrive

from geometric.molecule import Molecule as geometric_Molecule
from qcengine.compute import compute_procedure
from qcelemental.models import DriverEnum, Molecule
from qcelemental.models.procedures import TorsionDriveInput, \
    QCInputSpecification, OptimizationSpecification
from qcengine.procedures.torsiondrive import TorsionDriveProcedure
from qcelemental.models.common_models import Model

molecule = Molecule(
    symbols=["C", "C", "H", "H", "H", "H", "H", "H"],
    connectivity=[
        (0, 1, 1),
        (0, 2, 1),
        (0, 3, 1),
        (0, 4, 1),
        (1, 5, 1),
        (1, 6, 1),
        (1, 7, 1),
    ],
    geometry=[
        1.5403406836914142, -1.0173082391323545, 0.9312810207342496,
        4.0719763300123235, -0.0975682592642413, -0.02203578938791481,
        0.0002563605701713713, 0.0013953403968729996, 0.0011121160323317373,
        1.309831306165048, -3.03614919350581, 0.5491856718564878,
        1.3800394103640543, -0.7181256543708329, 2.9707878359388222,
        5.6120991748009645, -1.1161249890160694, 0.907991575289461,
        4.302418801484787, 1.921022388748466, 0.36057345099334853,
        4.232223312568672, -0.3961916040297606, -2.061588178357897]
)

result = compute_procedure(
    input_data=TorsionDriveInput(
        keywords={
            "dihedrals": [(2, 0, 1, 5)],
            "grid_spacing": [15]
        },
        input_specification=QCInputSpecification(
            driver=DriverEnum.gradient,
            model=Model(method="GFN2-xTB", basis=None)
        ),
        initial_molecule=[molecule,],
        optimization_spec=OptimizationSpecification(
            procedure="geomeTRIC",
            keywords={
                "program": "xtb",
            }
        )
    ),
    procedure="torsiondrive"
)

Expected behavior I got the error

TorsionDrive error at 165:
geomeTRIC run_json error:
Traceback (most recent call last):
  File "/Users/zhiyiwu/miniconda3_x86/envs/torsiondrive/lib/python3.9/site-packages/geometric/run_json.py", line 225, in geometric_run_json
    geometric.optimize.Optimize(coords, M, IC, engine, None, params)
  File "/Users/zhiyiwu/miniconda3_x86/envs/torsiondrive/lib/python3.9/site-packages/geometric/optimize.py", line 1331, in Optimize
    return optimizer.optimizeGeometry()
  File "/Users/zhiyiwu/miniconda3_x86/envs/torsiondrive/lib/python3.9/site-packages/geometric/optimize.py", line 1293, in optimizeGeometry
    self.calcEnergyForce()
  File "/Users/zhiyiwu/miniconda3_x86/envs/torsiondrive/lib/python3.9/site-packages/geometric/optimize.py", line 1002, in calcEnergyForce
    spcalc = self.engine.calc(self.X, self.dirname)
  File "/Users/zhiyiwu/miniconda3_x86/envs/torsiondrive/lib/python3.9/site-packages/geometric/engine.py", line 873, in calc
    return self.calc_new(coords, dirname)
  File "/Users/zhiyiwu/miniconda3_x86/envs/torsiondrive/lib/python3.9/site-packages/geometric/engine.py", line 865, in calc_new
    raise QCEngineAPIEngineError("QCEngineAPI computation did not execute correctly. Message: " + ret["error"]["error_message"])
geometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Resource Error: Program xtb is registered with QCEngine, but cannot be found.

The system is OSX 11.6.2 with intel CPU.

[jthorton]

Hi @xiki-tempula this probably needs documenting somewhere but you need to use the python interface for xtb which is a separate package called xtb-python. Try adding it to the environment using conda install xtb-python -c conda-forge.

[xiki-tempula]

@jthorton Thank you for the help. I was wondering if you know how to set the number of threads that xTB uses in this script? Thank you.

[jthorton]

We haven't needed to do this before, but I think this is what you are looking for https://xtb-docs.readthedocs.io/en/latest/setup.html?highlight=threads#parallelisation.

[xiki-tempula]

@jthorton Thank you. If there were to be a keyword to do this. Do you think it would be:

        optimization_spec=OptimizationSpecification(
            procedure="geomeTRIC",
            keywords={
                "threads": 16,
                "program": "xtb",
            }

[jthorton]

No, I don't think xtb will respect that see here for supported keywords, @awvwgk will be the best person to confirm how to set the number of threads. Note that this is the number of threads used per gradient calculation, the torsiondrive optimisations are not run in parallel currently.

[xiki-tempula]

@jthorton Thanks. I have raised an issue on xtb-python https://github.com/grimme-lab/xtb-python/issues/65 and I kind of want to make sure that the interface that I have purposed makes sense. I have noticed that the majority of time seems to be spent not on the xTB anyway.

[xiki-tempula]

@jthorton Thanks for the help.

From my experience with the original torsion drive, geomeTRIC is not a stable optimiser and I usually use the 'native-opt' routine of the torsion drive, which is just doing the geometry optimisation using the QM software is much more stable.

In this case, I wonder how do I ask QCEngine to do the optimisation with constrained dihedral using the xTB? Here is the current script.

result = compute_procedure(
    input_data=TorsionDriveInput(
        keywords={
            "dihedrals": [(2, 0, 1, 5)],
            "grid_spacing": [15]
        },
        input_specification=QCInputSpecification(
            driver=DriverEnum.gradient,
            model=Model(method="GFN2-xTB", basis=None)
        ),
        initial_molecule=[molecule,],
        optimization_spec=OptimizationSpecification(
            procedure="geomeTRIC",
            keywords={
                "program": "xtb",
            }
        )
    ),
    procedure="torsiondrive"
)

[jthorton]

In this case, I wonder how do I ask QCEngine to do the optimisation with constrained dihedral using the xTB?

Unfortunately, the native optimisation method is not currently supported by qcengine but has been talked about in the past and adding support should be possible and I am sure a PR would be welcome to implement it.

[xiki-tempula]

@jthorton Thank you. I wonder if procedure="geomeTRIC" is the only Optimization routine available? Sorry I cannot find the documentation for this.

[jthorton]

There are two others available pyberny and optking however I am not sure if they can handle constraints yet which are needed for torsiondrives.

[xiki-tempula]

@jthorton Thanks.