RuntimeError appeared when using MACE model for geometry optimization via ASE calculator

[HaoWan321]

Hi: my error message is: Traceback (most recent call last): File "mace_calc_md.py", line 28, in dyn.run(100) File "/u/hwan/.local/lib/python3.7/site-packages/ase/md/md.py", line 137, in run return Dynamics.run(self) File "/u/hwan/.local/lib/python3.7/site-packages/ase/optimize/optimize.py", line 156, in run for converged in Dynamics.irun(self): File "/u/hwan/.local/lib/python3.7/site-packages/ase/optimize/optimize.py", line 122, in irun self.atoms.get_forces() File "/u/hwan/.local/lib/python3.7/site-packages/ase/atoms.py", line 788, in get_forces forces = self._calc.get_forces(self) File "/u/hwan/.local/lib/python3.7/site-packages/ase/calculators/abc.py", line 23, in get_forces return self.get_property('forces', atoms) File "/u/hwan/.local/lib/python3.7/site-packages/ase/calculators/calculator.py", line 737, in get_property self.calculate(atoms, [name], system_changes) File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/mace/calculators/mace.py", line 63, in calculate out = self.model(batch) File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, *kwargs) File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/mace/modules/models.py", line 200, in forward node_e0 = self.atomic_energies_fn(data.node_attrs) File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(input, **kwargs) File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/mace/modules/blocks.py", line 72, in forward return torch.matmul(x, self.atomic_energies) RuntimeError: expected scalar type Float but found Double

My running script is: from ase import units from ase.io import read, write import numpy as np import time from ase.md.langevin import Langevin from mace.calculators import MACECalculator from ase.constraints import ExpCellFilter from ase.optimize.precon import PreconLBFGS from ase.optimize import QuasiNewton

model_path = '/u/hwan/IrOx/MACE_test/MACE_from_generation_with_1906_PT3/2806_run2/checkpoints/MACE_model_run-123.model' calculator = MACECalculator(model_path=model_path, device='cuda')

init_conf = read('./init.traj') out_file = 'init_R_Ir16O32_110_mace_opt' init_conf.set_pbc([True, True, False])

init_conf.set_calculator(calculator) opt = QuasiNewton(init_conf, logfile='%s.log' % out_file, trajectory = '%s.xyz' % out_file) # big cell, use preconditioned minimiser opt.run(fmax=1e-3)

for the MACE training, I have: python /raven/u/hwan/mace/scripts/run_train.py \ --name="MACE_model" \ --train_file="tmp_atoms_training.xyz" \ --valid_fraction=0.05 \ --test_file="tmp_atoms_testing_remove_configtype.xyz" \ --config_type_weights='{"Default":1.0}' \ --energy_key="DFT_energy" \ --forces_key="DFT_forces" \ --E0s='{8:-0.08969644, 77:-0.33524439}' \ --model="MACE" \ --hidden_irreps='128x0e + 128x1o' \ --r_max=5.0 \ --batch_size=30 \ --max_num_epochs=2000 \ --swa \ --start_swa=1600 \ --ema \ --ema_decay=0.99 \ --amsgrad \ --restart_latest \ --device=cuda \

Do you have any possible suggestions how does the RuntimeError comes from and how can I solve it?

[ilyes319]

Hey,

I am trying to understand this bug, I don't yet know what is causing it. Can you give me your torch version and the branch you are using?

Here is a simple fix that should make it work. Please apply that to your model.

model = torch.load(path_model)
model = model.double()
torch.save(model,path_model)

[HaoWan321]

my torch version is 1.12.0+cu102; for MACE I use main branch. I did as you wrote above: from ase import units from ase.io import read, write import numpy as np import time from ase.md.langevin import Langevin from mace.calculators import MACECalculator from ase.constraints import ExpCellFilter from ase.optimize.precon import PreconLBFGS from ase.optimize import QuasiNewton import torch

model_path = '/u/hwan/IrOx/MACE_test/MACE_from_generation_with_1906_PT3/2806_run2/checkpoints/MACE_model_run-123.model'

model_path = '/u/hwan/IrOx/MACE_test/MACE_from_test64/checkpoints/MACE_model_run-123.model' model = torch.load(model_path) model = model.double() torch.save(model, model_path)

calculator = MACECalculator(model_path=model_path, device='cuda')

init_conf = read('./init.traj') out_file = 'init_R_Ir16O32_110_mace_opt' init_conf.set_pbc([True, True, False])

init_conf.set_calculator(calculator) opt = QuasiNewton(init_conf, logfile='%s.log' % out_file, trajectory = '%s.xyz' % out_file) # big cell, use preconditioned minimiser opt.run(fmax=1e-3) I still have: File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/mace/modules/blocks.py", line 72, in forward return torch.matmul(x, self.atomic_energies) RuntimeError: expected scalar type Float but found Double

[ilyes319]

Can you add calculator = MACECalculator(model_path=model_path, device='cuda', dtype="float32")

`

[HaoWan321]

yes, now I did: calculator = MACECalculator(model_path=model_path, device='cuda', dtype="float32") also: calculator = MACECalculator(model_path=model_path, device='cuda', dtype="float64") Still: File "/u/hwan/conda-envs/mace_env/lib/python3.7/site-packages/mace/modules/blocks.py", line 72, in forward return torch.matmul(x, self.atomic_energies) RuntimeError: expected scalar type Float but found Double

[ilyes319]

Sorry for the delay, can you try the same thing but using the develop branch? Switch the calculator line to, calculator = MACECalculator(model_paths=[model_path], device='cuda', dtype="float32") If you are willing to send me your model by mail, I can see what is happening. This bug has been reported several times now but I have no idea what triggers it.

[davkovacs]

@HaoWan321 is this issue resolved?

[HaoWan321]

@HaoWan321 is this issue resolved?

I am testing now, I have been away for summer vacation and conference. Sorry for this later reply

[HaoWan321]

@HaoWan321 is this issue resolved?

I am testing now, I have been away for summer vacation and conference. Sorry for this later reply

Sorry for the delay, can you try the same thing but using the develop branch? Switch the calculator line to, calculator = MACECalculator(model_paths=[model_path], device='cuda', dtype="float32") If you are willing to send me your model by mail, I can see what is happening. This bug has been reported several times now but I have no idea what triggers it.

can u provide your email, maybe it be more efficient you send my model to u

[HaoWan321]

hello, the problem solved when I updated the MACE to the latest, the previous MACE I installed one year ago and now I use MACE_model_swa.model for ASE optimization works very well

[ilyes319]

Amazing, closing it!