peastman
commented
9 months ago Here's a first pass at a script for submitting datasets (not yet tested).
from qcportal import PortalClient
from qcportal.singlepoint import QCSpecification, SinglepointDatasetNewEntry
from qcportal.molecules import Molecule
from openmm.unit import nanometer, bohr
import openff.toolkit
import openff.units
import numpy as np
import h5py
import sys
dataset_name = sys.argv[1]
filename = sys.argv[2]
input_file = h5py.File(filename)
client = PortalClient.from_file()
scale = (1*nanometer).value_in_unit(bohr)
keywords = {'maxiter': 200,
'scf_properties': ['dipole', 'quadrupole', 'wiberg lowdin indices', 'mayer indices', 'mbis charges', 'mbis dipoles', 'mbis quadrupoles', 'mbis octupoles'],
'wcombine': False}
spec = QCSpecification(program='psi4', driver='gradient', method='wb97m-d3bj', basis='def2-tzvppd', keywords=keywords)
dataset = client.add_dataset('singlepoint', dataset_name)
dataset.add_specification('wb97m-d3bj/def2-tzvppd', spec)
for group in input_file:
smiles = input_file[group]['smiles'].asstr()[0]
conformations = np.array(input_file[group]['conformations'])*scale
ffmol = openff.toolkit.topology.Molecule.from_mapped_smiles(smiles, allow_undefined_stereo=True)
symbols = [atom.symbol for atom in ffmol.atoms]
total_charge = sum(atom.formal_charge/openff.units.unit.elementary_charge for atom in ffmol.atoms)
for conformation in conformations:
molecule = Molecule(symbols=symbols, geometry=conformation.flatten(), molecular_charge=total_charge, canonical_isomeric_explicit_hydrogen_mapped_smiles=smiles)
dataset.add_entry(group, molecule)
dataset.submit(tag='spice-psi4-181')Does that look generally correct? A first test will be to resubmit the DES monomers dataset with it. If everything is working correctly, QCArchive should recognize that all the samples are identical to ones that already exist and skip calculating anything. Assuming that works, the next test would be to force it to recompute them by adding a small offset to the positions and see if the results match what we got before.
j-wags
bennybp
loriab
dotsdl
jchodera
We need to figure out how we're going to submit calculations for SPICE 2. For version 1 we used the https://github.com/openforcefield/qca-dataset-submission repository. It uses github automation, so that you submit calculations by creating pull requests. Status updates are automatically posted to the PR, and it handles error cycling. The automation scripts use QCSubmit, which is a layer on top of the QCPortal API to provide additional useful features.
Due to a major redesign of QCPortal, QCSubmit is currently not functional. It's being updated, but there isn't a firm date for when the new version will be ready. Also, the new version of QCPortal provides many of the features it was created to add. According to @j-wags, QCSubmit still provides useful features for optimization datasets. For single point datasets, which is what we're using, the benefits are much smaller.
So we have a few questions to decide.
It looks to me like creating submissions should be about equally simple either way. Having status updates automatically posted to github is convenient but probably not that important. It's easy to write a script that you run to check the status. Error cycling might be more complicated, although I'm hoping it may no longer be as important. Newer versions of psi4 seem to be a lot better about not producing the sort of intermittent errors it's needed for. And while generating v1, we wasted a lot of computation time on error cycling, just repeating the same calculations over and over that kept failing.
Another benefit of the github approach is that it provides a record of the submission, but that too may be unnecessary for our purposes. All our submission will be the same type of dataset, and they'll all use exactly the same level of theory and other settings. I believe the only inputs needed for the submission will be the dataset name, and the HDF5 file containing the conformations (which is already in the repository).
cc @dotsdl