search

jbornschein / draw

Reimplementation of DRAW
MIT License
347 stars 84 forks source link

NLL bound high? #4

Closed udibr closed 9 years ago

udibr commented 9 years ago

After 100 epochs I got train_nll_bound 91.1 and test_nll_bound of 90.5 is that similar to what you got? I'm asking because the paper in Table 2 reported 80.97

ghost commented 9 years ago

Hmmm, .., not sure you're ever going to reproduce deepmind's number without their exact code?

There's too many unknowns in how they implemented it.

What do your generated samples look like? If they look good, I'd be content with that - it's not easy coding up that paper?

jbornschein commented 9 years ago

I never managed to train it down to ~81. But with some earlier version of this code I managed to train it down to a testset NLL of ~83 -- but I was training for more than 100 epochs.

I'll have to look carefully at my changes again to see if I changed anything that could degrade the results.

dribnet commented 9 years ago

I get test_nll below 90 in the first 50 epochs, and after a couple of hundred am below 83 - perhaps the blocks/fuel libraries have gotten better since this issue was created? So probably safe to now close.

As an example, below is a recent result from just over 300 epochs with test_nll=81.9 train_nll=81.8

samples