Reproducibility issue - Githubissues

yandex / rep

Machine Learning toolbox for Humans

http://yandex.github.io/rep/

Other

687 stars 144 forks source link

Reproducibility issue #5

Open tlikhomanenko opened 9 years ago

tlikhomanenko commented 9 years ago

All of neural nets (pybrain, neurolab, theanets) have problems with reproducibility:

pybrain https://github.com/pybrain/pybrain/issues/170
neurolab https://github.com/zueve/neurolab/issues/26
theanets https://github.com/lmjohns3/downhill/issues/3

Meanwhile there are some dirty hacks with changing global seed, which is awful. (And they don't work for pyBrain, so things there are totally irreproducible)

arogozhnikov commented 9 years ago

there is one more problem, which is common for all NNs - they aren't supporting weights, but in theanets this will be probably resolved.

tlikhomanenko commented 8 years ago

In theanets reproducibility issue is fixed in 0.7.3 by adding rng during Downhill.Dataset creating https://github.com/lmjohns3/theanets/blob/v0.7/theanets/graph.py#L327 and this fix https://github.com/lmjohns3/theanets/commit/94662f50800b5815c2ff97ab38b5488732eb9045.

This is shown in https://github.com/lmjohns3/theanets/issues/102.