search

CompPhysVienna / n2p2

n2p2 - A Neural Network Potential Package
https://compphysvienna.github.io/n2p2/
GNU General Public License v3.0
217 stars 82 forks source link

Implementing the (relative) error of forces as a cost function #128

Open Akram-Ibrahim opened 2 years ago

Akram-Ibrahim commented 2 years ago

Hi Dr. Singraber,

It is usually tough to fit a NNP to precisely reproduce the phonon dispersion curves especially if the system is quite complex. The current cost function implemented in n2p2 depends on the absolute error of forces which generates a NN that is biased toward a good fit for the forces of large magnitudes while it neglects the small absolute errors (but large relative errors) for the phonons.

I think might be beneficial if you can add a feature to choose between either to use absolute or relative errors for forces inside the cost function. This maybe also carried out to the Threshold updating criteria.

Thanks!

Akram-Ibrahim commented 2 years ago

I tried to make that change for my personal use but I do not know if I missed thing up.
I only changed this 2 code blocks in the Train.cpp file.

[1] image

[2] image

And maybe I need to add a regularizer in the denominator. I appreciate if you can comment on this.

singraber commented 2 years ago

Hello!

That's an interesting suggestion! As you already mentioned there are multiple spots where code changes will be required, in order of appearance in the Training::update() function:

  1. Threshold criterion: https://github.com/CompPhysVienna/n2p2/blob/6cc948bb052718b6d4225d62efb33b9d3abdef78/src/libnnptrain/Training.cpp#L1836-L1844

  2. Error derivative computation: https://github.com/CompPhysVienna/n2p2/blob/6cc948bb052718b6d4225d62efb33b9d3abdef78/src/libnnptrain/Training.cpp#L2018-L2025

  3. Another instance of (1): https://github.com/CompPhysVienna/n2p2/blob/6cc948bb052718b6d4225d62efb33b9d3abdef78/src/libnnptrain/Training.cpp#L2065-L2071

  4. Error compuation: https://github.com/CompPhysVienna/n2p2/blob/6cc948bb052718b6d4225d62efb33b9d3abdef78/src/libnnptrain/Training.cpp#L2105-L2109

  5. Error computation (per-element updater): The code paths with if (updateStrategy == US_COMBINED) are not used any more, they correspond to update_strategy 1, which does not work very well. So, if you don't care you can also omit this spot. https://github.com/CompPhysVienna/n2p2/blob/6cc948bb052718b6d4225d62efb33b9d3abdef78/src/libnnptrain/Training.cpp#L2126-L2132

And of course a regularizer should be added in the denominator as you already suggested. Could you please set up a pull request with the code changes (just like you already did), then I will add a new keyword for this option (maybe a second keyword for regularizer settings). After some testing we can of course merge this feature to the master branch. Thanks for your input!

All the best,

Andreas Singraber

Akram-Ibrahim commented 2 years ago

Hi Dr. Singraber,

I am very sorry for the late reply due to some family commitments. I will do that of course. Though, I do not understand what modification is needed for the second point you referred to for the error derivative computation.

Thank you!

Akram-Ibrahim commented 2 years ago

I'm very sorry. I also have one more related question. Does changing the cost function to penalize on relative force errors (in Training.cpp as you described above) transform the train/test RMSE screen output measures to relative RMSE as well? Or the performance measures are separated from the cost function changes and so they will remain absolute?