leo-p
commented
7 years ago Summary:
- Dataset: SVHN, MNIST, USPS, CIFAR and STL.
- Objective: Build a network easily trainable by back-propagation to perform unsupervised domain adaptation while at the same time learning a good embedding for both source and target domains.
Architecture:
Very similar to RevGrad but with some differences.
Basically a shared encoder and then a classifier and a reconstructor.
The two losses are:
- the usual cross-entropy with softmax for the classifier
- the pixel-wise squared loss for reconstruction
Which are then combined using a trade-off hyper-parameter between classification and reconstruction.
They also use data augmentation to generate additional training data during the supervised training using only geometrical deformation: translation, rotation, skewing, and scaling
Plus denoising to reconstruct clean inputs given their noisy counterparts (zero-masked noise and Gaussian noise).
Results:
Outperforms state of the art on most tasks at the time, now outperformed itself by Generate To Adapt on most tasks.
https://arxiv.org/pdf/1607.03516.pdf
In this paper, we propose a novel unsupervised domain adaptation algorithm based on deep learning for visual object recognition. Specifically, we design a new model called Deep Reconstruction-Classification Network (DRCN), which jointly learns a shared encoding representation for two tasks: i) supervised classification of labeled source data, and ii) unsupervised reconstruction of unlabeled target data.In this way, the learnt representation not only preserves discriminability, but also encodes useful information from the target domain. Our new DRCN model can be optimized by using backpropagation similarly as the standard neural networks. We evaluate the performance of DRCN on a series of cross-domain object recognition tasks, where DRCN provides a considerable improvement (up to ~8% in accuracy) over the prior state-of-the-art algorithms. Interestingly, we also observe that the reconstruction pipeline of DRCN transforms images from the source domain into images whose appearance resembles the target dataset. This suggests that DRCN's performance is due to constructing a single composite representation that encodes information about both the structure of target images and the classification of source images. Finally, we provide a formal analysis to justify the algorithm's objective in domain adaptation context.