sparks-baird / matbench-genmetrics

Generative materials benchmarking metrics, inspired by guacamol and CDVAE.
https://matbench-genmetrics.readthedocs.io/
MIT License
34 stars 2 forks source link

integration of energy above hull predictor #55

Closed sgbaird closed 2 years ago

sgbaird commented 2 years ago

Preferably one trained on both relaxed and unrelaxed structures, and one with minimal dependency issues. CDVAE does something similar, but the dependencies/build are too fragile at the moment to integrate into matbench-genmetrics.

Related to this, I came across an article saying effectively "don't discard structures just because they have a high energy above hull, e.g. over 200 meV/atom":

(1) Pöhls, J.-H.; Heyberger, M.; Mar, A. Comparison of Computational and Experimental Inorganic Crystal Structures. Journal of Solid State Chemistry 2020, 290, 121557. https://doi.org/10.1016/j.jssc.2020.121557.

Some options:

Something that can be pip and/or conda installed, has state-of-the-art predictive performance, and can be called with just a few lines of code:

from codebase import MyPretrainedModel
ehull_pred = MyPretrainedModel.predict(structures, property="e_above_hull")

in addition to the note above about being trained on both relaxed and unrelaxed structures. I think I could use the "MP + WBM" dataset mentioned by Rhys in https://github.com/materialsproject/matbench/issues/104#issuecomment-1030739336.

See also Data-Augmentation for Graph Neural Network Learning of the Relaxed Energies of Unrelaxed Structures

Other related repos:

sgbaird commented 2 years ago

There is also the question of whether this should be a separate implementation/repository and whether crystal generative benchmarks should depend on a property predictor.

kjappelbaum commented 2 years ago

I think this is a really tricky thing because you'll often be out of the domain on which these models have been trained on.

sgbaird commented 2 years ago

@kjappelbaum good point. Opens a whole new can of worms. Maybe we leave this out for now.