[Bug]: Error in prediction using finetuning model

[SmallBearC]

Email (Optional)

1499347301@qq.com

Version

0.7.1

Which OS(es) are you using?

• [ ] MacOS
• [ ] Windows
• [X] Linux

What happened?

1. I've Finetuning a pre-trained M3GNet, and It looks good, converges quickly, and has good accuracy，I have already uploaded the result image.
2. I used this model to predict, but the results were quite unexpected. The result image has been attached.
3. I don't know where the problem lies, because I have already trained myself from scratch many times and the process is exactly the same without any problems. The only difference now is that the pre trained model is loaded before training during finetuning. I have attached some of the code. The rest of the training is exactly the same as before.

Code snippet

Train：
    model = matgl.load_model('M3GNet-MP-2021.2.8-PES')
    lit_model = PotentialLightningModule(model=model.model, lr=0.00005, force_weight=1)

    trainer = pl.Trainer(max_epochs=10, accelerator='cuda', devices=1, precision='32')
    trainer.fit(model=lit_model, train_dataloaders=train_loader, val_dataloaders=val_loader)
    model_export_path = "./trained_model/"
    model.save(model_export_path)

Prediction：
if __name__ == '__main__':
    DB = Get_db()
    strus,e,f,sr = DB.get_stru_energy_forces_stress("vasprun.xml")
    print(f"{len(strus)} structures found !!! .")
    model =  matgl.load_model("../train/trained_model").model
    pre_e = e
    plt.scatter(range(1, len(strus)+1), pre_e, c='r')
    plt.scatter(range(1, len(strus)+1), energy, c='b')
    plt.savefig('res.png')
    plt.show()

Log output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[shyuep]

The problem is that you have a ".model" after the load_model. The pre-trained model has a transformer that operates on top of output from the model based on the mean and std deviation of the dataset. You are in effect bypassing the transformation. So what you are getting is a unscaled result and not the properly scaled one.

[SmallBearC]

Thank you very much for your reply, but I am still confused. When I was using it, following the example in your manual, I first imported the pre trained model ( PES_model ), then obtained the M3GNet model ( PES_model.model ), and use this M3GNet to train. Finally saved the M3GNet model. When I load it again (matgl. load_model()), it becomes an M3GNet model, and then I can use predict_structure function. I obtain a value that differs significantly from the training time. According to your reply, I am missing a scaling step, which results in this value not being the actual value. I would like to know how to avoid bypassing the scaling step you mentioned? Because I want to use it to predict some values to determine the ability to predict new structures. I really hope to receive your guidance. Thank you again

[kenko911]

Hi @SmallBearC, sorry for the late reply. I would suggest creating a Potential class and putting the calculated data_mean, data_std, element_refs and trained M3GNet model as input args. The next step is to create M3GNetCalculator to store the potential class and you can now calculate energies, forces and stresses with ASE atoms object. It should be noted that the predict_structure function is mainly for the property model since it doesn't include any gradient calculations (e.g forces, stresses, and hessian). Next time, you should do lit_model.model.save() and then the save model would be Potential class instead of M3GNet class. Please let me know if any further questions and I will close the issue. Thanks!

[SmallBearC]

Thank you very very much，it works now！

发自我的iPhone

------------------ Original ------------------ From: Tsz Wai Ko @.> Date: Mon,Sep 25,2023 4:01 AM To: materialsvirtuallab/matgl @.> Cc: SmallBearC @.>, Mention @.> Subject: Re: [materialsvirtuallab/matgl] [Bug]: Error in prediction usingfinetuning model (Issue #172)

Hi @SmallBearC, sorry for the late reply. I would suggest creating a Potential class and putting the calculated data_mean, data_std, element_refs and trained M3GNet model as input args. The next step is to create M3GNetCalculator to store the potential class and you can now calculate energies, forces and stresses. It should be noted that the predict_structure function is mainly for the property model since it doesn't include any gradient calculations (e.g forces, stresses, and hessian). Next time, you should do lit_model.model.save() and then the save model would be Potential class instead of M3GNet class. Please let me know if any further questions and I will close the issue. Thanks!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>