Weight.????.epoch Files

[Matheus-Novoa]

How i select the best weight file when i use nnp-train tool?

[owenrett]

Generally you select the epoch that has the lowest test RMSE (RMSE_Ftest_pu or RMSEpa_Etest_pu), as listed in the learning-curve.out file (lists the test/training RMSE files). This generally shows how well the model performs on data it wasn't trained on, so should give you an idea of how the model will perform on data it hasn't seen before.

[Matheus-Novoa]

I checked that file (learning-curve.out), but it gives me only first epoch results. learning-curve.out.txt

[owenrett]

Hmm, that's quite odd, could you post your nnp-train.log.0000 file? The same principle for selecting the correct epoch should apply there with respect to the F_test metric reported near the end of the nnp-train.log.0000 file. I had asked for the learning-curve.out.txt file as it contains more information and is easier to process.

[Matheus-Novoa]

nnp-train.log.0000.txt

[owenrett]

Odd that learning-curve.out.txt didn't update with the full training information, but in this case you'd likely select the last (10th) epoch as it has the lowest test force error (F_test = 9.17156E-02).

[Matheus-Novoa]

So was it a bug? Did i set something wrong in input.nn? What would a correct learning-curve.out file looks like? scaling.data.txt input.nn.txt

[owenrett]

Honestly I am unsure if it was a bug or not, I am only a user of n2p2 rather than a developer, going to let someone more knowledgeable take over if possible.

[singraber]

Indeed this is a strange behavior.. of course the learning-curve.out file is supposed to show the same epochs as the log file. Does that always happen or only once?

The final choice of weights is a non-trivial task.. the lowest test force error is certainly a good start but there are also other considerations like presence of over-fitting. Please see the general literature for machine learning about this topic. I guess the log file is only from a small test run.. there are only 20 structures in the data set and as expected the training and test errors deviate quite a lot while they should be comparable. Also I recommend to rerun the training with different initial random seeds which causes not only different initial weights but also a different split between training and test set. Hence you get a better understanding of the error to expect.