search

karpathy / char-rnn

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
11.51k stars 2.58k forks source link

Training determinism #17

Open sanxiyn opened 9 years ago

sanxiyn commented 9 years ago

It seems that running train.lua with same parameters is deterministic (I get exactly same train_bpc numbers), but -eval_val_every changes training. Why does how often validation is done affect training? Can it be made deterministic?

Is it because of RNG?

YafahEdelman commented 9 years ago

It is an RNG but the seed (starting state) stays the same between runs. You set the seed by adding -seed [some number]. The default seed is 123.

sanxiyn commented 9 years ago

My question was that the same seed produces different results, if validation frequency is changed.

karpathy commented 9 years ago

Sorry for late reply on this. This seems odd because the eval code does not use the RNG as far as I can tell. It seems it shouldn't have effect. I'll have a closer look