ACEsuit / mace

MACE - Fast and accurate machine learning interatomic potentials with higher order equivariant message passing.
Other
536 stars 198 forks source link

Can we train the periodic and nonperiodic structures together in the training set ? #580

Closed jinfeng-data closed 2 months ago

jinfeng-data commented 2 months ago

Hi, I train a mace model using the periodic and nonperiodic structures together in the training set. When I use the trained model to perform the md simulations, it can run normally under the nonperiodic boundary conditions. But when I move to the periodic boundary conditions, the md simulation is always not stable, with the temperature rising all the time. I want to know, is it ok that we train the periodic and nonperiodic structures together in the training set? or do we need to add more periodic structures to the training set ? Thanks in advance.

gabor1 commented 2 months ago

This should definitely not happen, and we have not seen it elsewhere. In order to help you debug, we need to see your training input, the log file of the training, a sample of your data (both periodic and non periodic) and the failing MD. If you don't want to share these publicly here, get in touch by email

jinfeng-data commented 2 months ago

This should definitely not happen, and we have not seen it elsewhere. In order to help you debug, we need to see your training input, the log file of the training, a sample of your data (both periodic and non periodic) and the failing MD. If you don't want to share these publicly here, get in touch by email protein_run-123.log MDsimulation.pos_0.txt run_train.txt trainingset_sample.txt

Hi, thanks for your reply. I have attached the necessary files. A mini training set sample is in trainingset_sample.xyz, which contains periodic and nonperiodic structures. The training input is in run_train.sh, the log is in protein_run-123.log, and the failed md output is in MDsimulation.pos_0.xyz. The failed MD was run under NVT ensemble with periodic boundary conditions. However, if I run with nonperiodic boundary conditions (i.e. using same molecular structures but with a super large size box), it can run stably and normally. Thanks for your time, and I am looking forward to your reply.

gabor1 commented 2 months ago

what are you using to run the MD ? bonds are starting to break right from the beginning, the trajectory is complete garbage, it's not a question of "stability". can you upload a trajectory with exactly the same input (cube of water) but with no PBC ?

gabor1 commented 2 months ago

(I don't think this is your problem, but I see that you are not using the latest foundation model - it's called mp0b, it's the latest tagged version in the mace-mp repo, it has pair repulsion built-in, so I would always start from that)

gabor1 commented 2 months ago

as a quick check, can you please run the periodic (and non periodic) md with exactly the same initial conditions as this one, but use the foundation model rather than the one you trained (either the one you started form or the mp0b one I mentioned above)

jinfeng-data commented 2 months ago

what are you using to run the MD ? bonds are starting to break right from the beginning, the trajectory is complete garbage, it's not a question of "stability". can you upload a trajectory with exactly the same input (cube of water) but with no PBC ?

simulation.pos_0.txt A trajectory with exactly the same input but with no PBC has been attached. I run MD by using i-PI program as following,

!/bin/bash

BSUB -gpu "mode=exclusive_process"

BSUB -n 1

BSUB -q gpu2

BSUB -o %J.out

BSUB -e %J.err

BSUB -J md

BSUB -R "rusage[ngpus_physical=1]"

cuda

export CUDA_HOME=/data/home/xiaohe/yuhaochen/software/cuda-11.8/bin export LD_LIBRARY_PATH=/data/home/xiaohe/yuhaochen/software/cuda-11.8/lib64 export PATH=$PATH:$LD_LIBRARY_PATH:$CUDA_HOME

source /data/home/xiaohe/jinfeng/i-pi-master/env.sh source /data/home/xiaohe/jinfeng/soft/mace-venv/bin/activate

HOST=$(hostname)

if [ -e simulation.restart ]; then sed -i "s/address>[^<]$HOST</" simulation.restart i-pi simulation.restart >> log.ipi & else sed -i "s/address>[^<]$HOST$HOST</" input.xml i-pi input.xml > log.ipi & fi sleep 3

sed -i "s/(host = \")[^\"]*/\1$HOST$HOST/" run.py sleep 3

python run.py > out

where the run.py is as the following,

from ase.io import read from mace.calculators import mace_mp from ase.calculators.socketio import SocketClient

from ase.calculators.socketio import SocketServer

Define atoms object

atoms = read("init.xyz", 0)

Set ASE calculator

macemp = mace_mp(model="../model0_multigpus/protein_stagetwo_compiled.model", device='cuda',default_dtype="float32") # return the default medium ASE calculator equivalent to mace_mp(model="medium") atoms.calc = macemp

Create Client

host = "gpu10.idm.hsdedu9.comgpu10.idm.hsdedu9.com" client = SocketClient(unixsocket=host) client.run(atoms, use_stress=True)

gabor1 commented 2 months ago

another cross-check that you could do is to simply run it entirely within ASE, using DynamicalSystems. I think the problem is somewhere between the ASE calculator and I-PI

jinfeng-data commented 2 months ago

as a quick check, can you please run the periodic (and non periodic) md with exactly the same initial conditions as this one, but use the foundation model rather than the one you trained (either the one you started form or the mp0b one I mentioned above)

I test the periodic md by using the foundation model, and the system also becomes garbage immediately at the beginning, which is like the case of using my trained model.

gabor1 commented 2 months ago

ok, so that suggests that it's not the model (because we know that the foundation model is quite happy to run water boxes), but your setup, likely some confusion between ASE and I-PI. Try the pure ASE dynamics to confirm. I'll ask an I-PI expert to comment too.

venkatkapil24 commented 2 months ago

Hi, the i-PI developer here. Can you share the input.xml file and the configuration that you are using to initialize the i-PI?

jinfeng-data commented 2 months ago

Hi, the i-PI developer here. Can you share the input.xml file and the configuration that you are using to initialize the i-PI?

init.txt input.txt

Hi, thanks for your reply, and the input.xml and init.xyz have been attached.

venkatkapil24 commented 2 months ago

can you elaborate "the system also becomes garbage immediately at the beginning"? What property are you studying and by immediately do you mean like the first 10 steps or do you mean more. It would really help to see your simulation.out file with the mace-mp-0 model. In so far for 100 steps I have stable temperature control and no signs of instability.

jinfeng-data commented 2 months ago

simulation.pos_0.txt simulation.out.txt

My simulation.out and the trajectory file have been attached. I want to have stable md simulations.

jinfeng-data commented 2 months ago

ok, so that suggests that it's not the model (because we know that the foundation model is quite happy to run water boxes), but your setup, likely some confusion between ASE and I-PI. Try the pure ASE dynamics to confirm. I'll ask an I-PI expert to comment too.

I use ASE dynamics, and it works. The md run is safe and stable under PBC. So it is the problem of i-PI setting, which lead to the md failure.

jinfeng-data commented 2 months ago

can you elaborate "the system also becomes garbage immediately at the beginning"? What property are you studying and by immediately do you mean like the first 10 steps or do you mean more. It would really help to see your simulation.out file with the mace-mp-0 model. In so far for 100 steps I have stable temperature control and no signs of instability.

Can I have your input scripts for md run with i-PI ? I want to check whether my scripts are correct.

venkatkapil24 commented 2 months ago

Hi, I have not been able to reproduce your unstable simulations with MACE-MP-0. My simulations are fine. I am using the same input file as you were but here are the input and output files. run-ase.py.txt simulation.out.txt input.xml.txt

jinfeng-data commented 2 months ago

I find it is the problem of my previously installed i-PI program. I reinstall the latest version of i-PI, and now it works. Thank you all !

ilyes319 commented 2 months ago

Ok closing then.