git clone git@github.com:InfluenceFunctional/MXtalTools.git MXtalTools
mxtaltools-env.yaml
in the downloaded MXtalTools codebase directory. Edit to correspond to your CUDA version. Then at the codebase directory, run
conda env create -f mxtaltools-env.yaml
on command line to create a conda virutal environment with the required packages. Alternatively, do
conda env create -f mxtaltools-env-simple-cpu.yaml
for a more bare-bones environment e.g., for small local inference. Activate the environment with
conda activate mxtaltools-env
Login to your weights and biases ("wandb") account, which is necessary for run monitoring and reporting with
wandb login
on the terminal command line.
--user YOUR_USERNAME
This software generates training datasets of molecular crystal structures from collections of .cif files. .cifs are collated and processed primarily with the CSD Python API and RDKit. Collation includes filtering of structures which are somehow invalid. Invalid conditions include: symmetry features disagree, no atoms in the crystal, RDKit rejects the structure outright. The Cambridge Structural Database (CSD) can be processed by first dumping it to .cif files, or directly with minor modifications. Customized functions are available for processing CSD Blind Test submissions TODO clean & test.
In the most common case, processing the CSD, to generate a dataset, run the following scripts,
dump_csd.py
--> cif_processor.py
--> manager.py
,
with the appropriate paths set in each script.
cif_processor.py
takes on the order of dozens of hours to process the full CSD (>1M crystals).
manager.py
also may take a few minutes to process a large dataset, as this is where we do pose analysis,
duplicates search, and some indexing tasks.
We recommend running several instances in parallel to reduce this time.
As they process datasets chunkwise in random order, this parallelism is fairly efficient.
Note that the speed here depends strongly on disk read-write speed.
The list of featurzed computed for each atom/molecule/crystal are listed in TODO generate list.
If you would like to use a different feature set, you may edit dataset_management/featurization_utils.py
,
or, if your features are fast to compute, add them at runtime in dataset_management/modelling_utils.py
.
Subsets of features, can be selected in the dataset config used for a given run in configs/dataset
.
Likewise, filters can be applied to the dataset to limit the crystals included.
Such filters may commonly include: molecule size, crystal setting, atomic number.
We also have functions for identifying duplicate molecules and polymorphs of the same molecule. When filtering these, we identify all the duplicates and pick a single 'representative' sample at random.
There are multiple modes for training, inference and analysis.
train_crystal_models
Within training mode, there are two types of models which can currently be trained.
Modes and datasets can be called via experiment configs and dataset configs.
configs/experiments
and called on the command line or in the user config)
see configs/test_configs/discriminator.yaml
.
This loads the full_gan specified dataset, and trains a discriminator model only on randn and distorted fakes.configs/dataset/full_gan.yaml
and configs/dataset/skinny_regression.yaml
.
These construct training datasets for training generator/discriminator models with all atom & molecule features, and
a regressor model with only a few input features.
Note the differences in filter conditions between modes, such as the filtering of all polymorphs and duplicate molecules
in the regression task, and the filtering of dubious or nonstandard settings in the generation task. bash_sub.sh
.