search

nishnik / ANONS

ANOVA for neural networks
4 stars 11 forks source link

added mnist implementation in pytorch and few plots #2

Closed pswaldia closed 5 years ago

nishnik commented 5 years ago

Thanks for the PR. LGTM, but few issues: You have to train a separate model for each of the hyper-parameter.

And log them in a .MD file, like done here

Keep replying here!

pswaldia commented 5 years ago

How many layers we need to consider.? Considering a simple dataset like MNIST taking more than 2 makes the test accuracy go to 100.00 for almost all epochs.

nishnik commented 5 years ago

Let's take 2 only.

pswaldia commented 5 years ago

Done with the changes! please review

nishnik commented 5 years ago

@pswaldia You misunderstood me, we don't want the accuracy for different activation functions. But let's keep the activation function fixed and log the accuracy for different hyperparameters.

  1. Learning rate 0.01 a. 3 layers(784->256->64->10) b. 2 layers(784->64->10)
  2. Learning rate 0.05 a. 3 layers(784->256->64->10) b. 2 layers(784->64->10)
pswaldia commented 5 years ago

i made the changes, keeping the activations functions fixed and varying learning rate and momentum for adam as an optimizer. It is done with neural network having 3 and 2 hidden layers. Please review

nishnik commented 5 years ago

This looks good to me. I am merging this, could you please do this for the four(2*2) combination:

  1. learning rate of 0.01 and 0.1
  2. dropout rate of 0.2 and 0.7 on cifar dataset, then we can use the results to do a small ANOVA test.

Run just for five epochs.

pswaldia commented 5 years ago

Yes I'll do that!