richardtomsett
commented
6 years ago From previous review: Erhan et al. (2009) developed one of the first methods for visualizing the responses of individual units in (unsupervised) deep belief networks. They developed methods for analyzing units in any layer of a network, while previous methods only looked at units in the first (input) layer.
Visualizing Higher-Layer Features of a Deep Network Deep architectures have demonstrated state-of-the-art results in a variety of settings, especially with vision datasets. Beyond the model definitions and the quantitative analyses, there is a need for qualitative comparisons of the solutions learned by various deep architectures. The goal of this paper is to find good qualitative interpretations of high level features represented by such models. To this end, we contrast and compare several techniques applied on Stacked Denoising Autoencoders and Deep Belief Networks, trained on several vision datasets. We show that, perhaps counter-intuitively, such interpretation is possible at the unit level, that it is simple to accomplish and that the results are consistent across various techniques. We hope that such techniques will allow researchers in deep architectures to understand more of how and why deep architectures work.
Bibtex:
@TECHREPORT{visualization_techreport, author = {Erhan, Dumitru and Bengio, Yoshua and Courville, Aaron and Vincent, Pascal}, month = jun, title = {Visualizing Higher-Layer Features of a Deep Network}, number = {1341}, year = {2009}, institution = {University of Montreal}, note = {Also presented at the ICML 2009 Workshop on Learning Feature Hierarchies, Montr{\'{e}}al, Canada.} }