nnp-comp2 usage

[Matheus-Novoa]

I had some doubts about the "Extrapolation problems" issue.

1. How I set the , , variables?

2. Which does settings it make sense modify in NNP1 and NNP2?

3. In the example folder, the only differences between the two input.nn files were that in one of them some symmetry function lines were commented out and in the other they were not. Are these modifications enough to distinguish one NNP from another?

4. Are there other settings that can be changed?

[owenrett]

Disclaimer: not a developer, so answer here is just based off of experience using the tools.

1)How I set the , , variables?

You call the function nnp-comp2 by first running nnp-comp2 compare

to get predictions from the two neural networks. Then you can specify the ew_threshold, energy_threshold, force_threshold parameters by calling

nnp-comp2 select <ew_threshold> <energy_threshold> <force_threshold> <species_name_1> <species_name_2> ...

where ew_threshold is the max number of extrapolation warnings before selecting a structure, energy_threshold is the max energy difference between two structures (on a per-atom basis), force_threshold is the max difference between any two predicted forces, and species_name_1, species_name_2,.. are the elements present.

3) In the example folder, the only differences between the two input.nn files were that in one of them some symmetry function lines were commented out and in the other they were not. Are these modifications enough to distinguish one NNP from another?

When using nnp-comp2 to perform active learning / generate new training data you want to keep all elements of the two networks the same except the random_seed to initialize the weights. This ensures that any differences in predictions between the two networks is due to this random initialization, not network structure differences. In this way, the structures where large differences exist, and are selected by nnp-comp2, are areas where the model is learning poorly. Any other changed parameters could mean that differences are not due to poor learning.

See Artrith, Nongnuch, and Jörg Behler. "High-dimensional neural network potentials for metal surfaces: A prototype study for copper." Physical Review B 85.4 (2012): 045439.

for an early example of this approach.

[Matheus-Novoa]

"... ew_threshold is the max number of extrapolation warnings before selecting a structure ..." So, in this example the the ew_threshold would be 1374 ?

As for energy_threshold and force_threshold, the difference that you mentioned, is it about the difference between the NNP prediction and the reference data, or between the two NNP predictions?

[owenrett]

Generally the ew_threshold should be set to the maximum number of extrapolation warnings you think is acceptable in your potential (generally speaking this is 0), and it will automatically select any structure with more than that number of extrapolation warnings.

The energy_threshold and force_threshold are the difference between the two NNP predictions, as the model does not strictly have access to reference data here. That said, the difference between two NNP predictions does have a correlation with the difference between one NNP prediction and a hypothetical DFT calculation at the same point.

Edit: Should note that energy_threshold is the difference between the average atomic energy between the two systems, and force_threshold is the difference between any two forces.

---

As an aside, if you provide the model with too diverse of data (very unphysical configurations), you can get issues where the model is unable to train effectively; trying to explain the very large energy and force difference between a config where atoms are 0.01 A apart and a config where atoms are 0.02 A apart is very difficult for a NNP to be frank. There's some recent work that puts an upper bound on the max force/energy difference for an eligible configuration instead of just a lower bound, however the reference escapes me currently. Will edit it in if I find it.

Huang, Jianxing, et al. "Deep potential generation scheme and simulation protocol for the Li10GeP2S12-type superionic conductors." The Journal of Chemical Physics 154.9 (2021): 094703.

[Matheus-Novoa]

I have one more question about how FDiffMean and FDiffMax parameters are computed. Is the difference computed on each component of each atom of each structure, or is some kind of normalization done?

[owenrett]

FDiffMean is the average difference between any two force predictions (from the two NNPs, for a given atom in the structure) across a given structure. FDiffMax is the maximum difference between any two force predictions (from the two NNPs, for a given atom in the structure) for a given structure.

eg if you had:

NNP1: 12, 3, 4 NNP2: 10, 3, 0

FDiffMean would be 2 FDiffMax would be 4

Apologies for poor formatting, hard to create a table to display this correctly here.

Edit: I should also note, each atom has three force components attached to it, aka F_x, F_y, F_z.

[Matheus-Novoa]

Just to clarify, in your example, 12 3 4 are F_x, F_y, F_z, right?