search

CompPhysVienna / n2p2

n2p2 - A Neural Network Potential Package
https://compphysvienna.github.io/n2p2/
GNU General Public License v3.0
217 stars 82 forks source link

How to update the NNP model with new data? #127

Closed Akram-Ibrahim closed 2 years ago

Akram-Ibrahim commented 2 years ago

The question Having a model fitted to a large dataset, how can I update the weights of this model by training it only on new extra data points (instead of repeating the training on the whole dataset)?

What did you already try? I tried using input.nn, scaling.data, and weights.???.data files from the up-to-date model and created a new input.data file with the new dataset only. After running the code by nnp-scaling and nnp-train, the original scaling.data file is replaced with that for the new dataset and the nnp-train exits with the error 'std::out_of_range'.

singraber commented 2 years ago

Unfortunately n2p2 offers no way of re-training only portions of the data set. Although there is in principle the option to start from previously optimized weights by setting the use_old_weights_short keyword in input.nn I would strongly advise against using this feature in combination with a new, reduced data set. Here is why:

  1. The way you already tried it hints at one of the problems: if you have new data this will likely change the symmetry function scaling. Hence the input neurons (whose values depend on the scaling information) will not match with the old weights to get the training results for the old data.

  2. Starting the weight optimization from an old local error minimum is not a good idea. It may be difficult for the optimization algorithm to "get out" of this local minimum and find a different solution for the new data.

  3. The weight optimization algorithm should be able to choose training patterns (energies and forces) from the entire set to get a good error balance. If you restart with only a portion of the entire data the optimizer may "focus" on the new data and neglects the old one.

Akram-Ibrahim commented 2 years ago

Thank you Dr. @singraber for the detailed reply. I used the ### use_old_weights_short keyword before in case I needed to do more epochs but in case of adding new data this may not be a good way to go as you mentioned (especially that in my case this new data is supposed to sample a different part of the PES other than the initial dataset so the symm func scaling is different and the model may be vulnerable to the recency effect) .

Thanks!