search

mzjb / DeepH-pack

Deep neural networks for density functional theory Hamiltonian.
GNU Lesser General Public License v3.0
246 stars 51 forks source link

question on the train paramter #35

Closed JTaozhang closed 1 year ago

JTaozhang commented 1 year ago

To developer,

When I use the examples of Bi and graphene system, I notice some parameters (revert_then_decay, revert_decay_epoch, revert_decay_gamma) in the train.ini file, which are not explained in the manual wbsite. Therefore,if we start a calculation on new system, I want to ask whether these parameters need to be considered.

best regard.

mzjb commented 1 year ago

Hi there,

The parameters you mentioned (revert_then_decay, revert_decay_epoch, revert_decay_gamma) are used during the training process to enhance stability. When the loss sharply increases, these parameters enable the neural network to revert back to an earlier state and decrease the learning rate to continue training, which can help to increase the stability of the training process.

If you are starting a new training process, you can leave these parameters at their default values. However, if you find that the learning rate is decreasing too quickly and leading to a large final loss, you can increase both the revert_decay_epoch and epochs parameters. Alternatively, you can increase the number of steps at which the learning rate decreases, for example by setting

[train]
epochs = 7000
revert_then_decay = True
revert_threshold = 60
revert_decay_epoch = [800, 2000, 4000, 6000]
revert_decay_gamma = [0.4, 0.5, 0.5, 0.5]
JTaozhang commented 1 year ago

ok. Thanks for your explanation, I will test them when I meet this situation.