search

jiaor17 / DiffCSP

[NeurIPS 2023] The implementation for the paper "Crystal Structure Prediction by Joint Equivariant Diffusion"
MIT License
82 stars 33 forks source link

A question on the value of losses #19

Open SorenGioro opened 3 months ago

SorenGioro commented 3 months ago

Hello JiaoR, Thanks for your great work! I’m currently using some slabs (nearly 3000) in the OpenCatalyst dataset training a model, and both val_lattice_loss and val_coord_loss are nearly 0.6 with default hyperparameters I wonder whether this value is reliable, could you please give me some advice?

jiaor17 commented 3 months ago

Hi,

Thanks for your interest!

Your reported validation losses (val_lattice_loss and val_coord_loss around 0.6) are indeed in line with what we have observed on MPTS-52. This suggests that your model is training as expected and the loss values are reliable.

However, it's important to note that validation loss values alone might not fully capture the model's performance in practical scenarios. To get a more comprehensive understanding of how well your model is performing, I recommend conducting a generation test. You can do this by running the generation process on a subset of your validation set and then calculating the match rate. This will provide a clearer picture of how your model performs on this dataset.

SorenGioro commented 3 months ago

Thank you for your reply! I will try it out later