Cleaned up examples a little

wuaalb commented 8 years ago

Fixes a number of small issues with the examples

Inconsistent output folders across different examples
Some help strings for argparse were not very helpful
Some options (like number of training epochs) were not settable through command line
In the mnist_ladder example, default value for command line argument -initval was 'relu' while it should be a numeric value which depends on -init (changed to 'None' meaning Lasagne default)
Some comments regarding tensor sizes were incorrect or inconsistent
Reduced batch size when estimating test log likelihood using 5k point importance sampling, to reduce memory requirements on smaller GPUs (e.g. 5000 (iw_samples) x 1 (eq_samples) x 50 (batch_size) x 784 (num_features) x 4 bytes is approx. 748 MB)
Actual number of epochs trained would be one less than specified in some examples
Added input shuffling for vae_vanilla training like other examples
Some minor clean ups

wuaalb commented 8 years ago

The preprocesses_dataset functions were swapped
Initial commit re-binarized for each epoch in vae_vanilla, now fixed

casperkaae commented 8 years ago

@wuaalb

Thanks again for the fixes :). I had to make a few changes to the examples to fix #18 (in PR #19). Do you want to rebase or should i merge the changes?

casperkaae commented 8 years ago

Actually #18 didn't fix the bug - should be fixed in this commit

wuaalb commented 8 years ago

I rebased with the latest changes.. but I think that latest commit is wrong..

preprocesses_dataset = lambda bernoullisample: bernoullisample

should be

preprocesses_dataset = bernoullisample

otherwise it does nothing.. Thanks :)

casperkaae commented 8 years ago

Yes of course :)

PR #20 seems to fix the performance. Sorry for keeping changing the examples - i've just wanted them to run properly. Can I ask you to rebase again (sorry again!)

I've changed the initialisation in the Ladder code - i've think it should stay the way it is now.

Thanks for the help

wuaalb commented 8 years ago

Re-re-rebased :)

The changes I made to the Ladder initializer cli argument handling fixes a different problem than the one you fixed. I rewrote and commented that part, so maybe it is clearer what it does now. I kept 'uniform' as an option.. wasn't sure if you removed it on purpose or not.

casperkaae commented 8 years ago

great - seems fine with me.

merged #17

casperkaae / parmesan

Cleaned up examples a little #17