search

corenel / pytorch-adda

A PyTorch implementation for Adversarial Discriminative Domain Adaptation
MIT License
474 stars 139 forks source link

Thanks ! + testing the "src_only" baseline ... #7

Open emmanuelrouxfr opened 6 years ago

emmanuelrouxfr commented 6 years ago

Hello, recently I've been working with your PyTorch implementation of ADDA so first of all thanks for your code!

For now I am only interested in testing the network Ms (trained on the source domain) on the target domain (src_only baseline):

Surprisingly on your github page you announced ~84% accuracy of this src_only baseline which is around 9% over the reported accuracy (75.2%) in the original paper (https://arxiv.org/abs/1702.05464). How can you explain such a difference?

I have tried to limit the number of samples in source domain (MNIST) to 2000 (as in the original paper) and yet I observed a ~87% accuracy (the last modifications I made from your master branch corresponds to that experiment : https://github.com/emmanuelrouxfr/pytorch-adda)

To be as close as possible from the paper setup, I have also tried to set the original batch size to 128 (adjusting the number of epochs to be 625 to fit the mentioned 10 000 learning iterations) and the original learning rates and parameters: d_learning_rate = 2e-4 c_learning_rate = 2e-4 beta1 = 0.5 beta2 = 0.999

but I can't reproduce the results originally presented in the original paper (~75% accuracy of src_only tested on USPS). It is always much higher than it is supposed to be.

I hope you could help me identify a possible reason to this phenomenon, thanks !

JinmingZhao commented 6 years ago

Hi，I use the code and get the results: === Evaluating classifier for encoded target domain ===

source only <<< Avg Loss = 0.3980948992383977, Avg Accuracy = 96.290323% domain adaption <<< Avg Loss = 0.41198907047510147, Avg Accuracy = 97.258065%

By modify the config: batch_size = 128 num_epochs = 1000 And it works.

emmanuelrouxfr commented 6 years ago

Hi @JinmingZhao ! It works but the "source only" baseline (you got 96.290323 %) is much higher than in the original paper (around 75 %) which is the issue I raised. Anyway, thanks you ! Bye bye

aasharma90 commented 5 years ago

@emmanuelrouxfr I also have the same concern - how come the results (even the baseline) here are better than what the ADDA's authors have reported in their main paper?