search

musyoku / adversarial-autoencoder

Chainer implementation of adversarial autoencoder (AAE)
258 stars 76 forks source link

Can't get the correct error rates #7

Closed gagb closed 7 years ago

gagb commented 7 years ago

Hey

For the semi-supervised version (100 labels), the current version gets around 85% accuracy. Which is different from the graph in the readme.

Here is the resultant semi-supervised/dim_reduction/results.csv result.txt

The performance of the unsupervised dimensionality reduction also seems to have decreased. Here's the visualization using the current version. The separation is worse than the visualization shown in readme. The error rate on the training data for this visualization is ~40%. The error rate reported in the original paper is ~13.5%. labeled_z_test_default

Thanks!

musyoku commented 7 years ago

Hi. I found that the default hyperparameter (learning_rate=0.001, momentum=0.5) is too high for semi-supervised / unsupervised learning.

/semi-supervised/classification/

learning_rate=0.001, momentum=0.9

image

learning_rate=0.001, momentum=0.1

image

learning_rate=0.0001, momentum=0.1

image

/unsupervised/dim_reduction/

learning_rate=0.0001, momentum=0.1, seed=2

2

learning_rate=0.0001, momentum=0.1, seed=3

3

learning_rate=0.0001, momentum=0.1, seed=4

4

I fixed the code.

Thank you :smiley:

gagb commented 7 years ago

The unsupervised version now gets ~28.5% classification error. Thanks! I am closing this issue for now because the lower performance seems to be because of choice of hyper-parameters and not because of an implementation error.