Closed ale99WGiais closed 5 months ago
The MACE numbers are in eV and eV/A, but the original dataset is in kcal/mol. Did you make the conversion? For numerical precision, it is better to use eV and eV/A in the MACE code.
Hi Ilyes, thanks for your very quick response!
No, I'm sorry but we didn't notice that the original dataset was in in kcal/mol.
We'll try converting the dataset to eV and refit the potentials asap :)
Describe the bug We tried to fit MACE potentials on some datasets mentioned in the reference paper "Evaluation of the MACE Force Field Architecture: from Medicinal Chemistry to Materials Science".
In particular we tried fitting MACE on "Double-walled nanotube" and "Buckyball catcher".
The MAE metrics obtained by us are very different from the ones stated in the paper, so we are wondering what we colud be doing wrong :(
To Reproduce
MACE was installed using the following commands
To fit MACE on the nanotube we used the following scripts:
According with the examples in https://mace-docs.readthedocs.io/en/latest/examples/training_examples.html
Similar scripts were adopted for the buckyball catcher.
The code was submitted to single Nvida Tesla A100 GPU machines with a time limit of about 3 days.
Data for both nanotube and buckyball was downloaded from here: http://www.sgdml.org/
Expected behavior
We expected to have low energy and force MAE as in the paper:
But we got errors orders of magnitude higher:
buckyball mace-256-0-r6-int1 stdout.txt buckyball mace-256-2-r3-int2 stdout.txt buckyball mace-256-2-r5-int2 stdout.txt nanotube mace-256-0-r6-int1 stdout.txt nanotube mace-256-2-r3-int2 stdout.txt nanotube mace-256-2-r5-int2 stdout.txt
Everything is uploaded here: https://uniudamce-my.sharepoint.com/:f:/g/personal/142135_spes_uniud_it/EvqCwMiR9PNMkZqb8L5iQTMBnHnpEm0-CQVCOsEskxbdaA?e=dhfnXJ
Thanks very much for the support, Alessio