Closed jiunbae closed 5 years ago
Abstract: Propose a new deep unsupervised hashing function, called HashGAN, which efficiently obtains binary representation of input images without any supervised pretraining, consists of generator, discriminator and encoder.
Proposing framework for unsupervised hashing model coupling deep hash function and GAN.
Sharing parameters of the encoder and discriminator, get benefit from the adversarial loss as a data-dependent regularization in training our deep hash function.
Hashing loss function for real images, results in minimum entropy, uniform frequency, consistent and independent hash bits.
Employ collaborative loss in training, enforcing similar random inputs and hash bits for synthesized images.
they require costly human-annotated labels to train their large set of parameters.
Supervised hash functions: 16, 30, 33, 52
W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang. Supervised hashing with kernels. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2074–2081. IEEE, 2012.
W.-J. Li, S. Wang, and W.-C. Kang. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 1711–1717. AAAI Press, 2016.
J. Guo, S. Zhang, and J. Li. Hash learning with convolutional neural networks for semantic based image retrieval. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 227–238. Springer, 2016.
R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan. Supervised hashing for image retrieval via image representation learning. In AAAI, volume 1, pages 2156–2162, 2014.
Deep hash function, supervised:, 8, 27, 55, 59
T.-T. Do, A.-D. Doan, and N.-M. Cheung. Learning to hash with binary deep neural network. In European Conference on Computer Vision (ECCV), pages 219–234. Springer, 2016.
H. Lai, Y. Pan, Y. Liu, and S. Yan. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3270–3278, 2015.
H.-F. Yang, K. Lin, and C.-S. Chen. Supervised learning of semantics-preserving hash via deep convolutional neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
H. Zhu, M. Long, J. Wang, and Y. Cao. Deep hashing network for efficient similarity retrieval. In AAAI, pages 2415–2421, 2016.
Unsupervised hash functions: 18, 19, 46, 50
K. He, F. Wen, and J. Sun. K-means hashing: An affinitypreserving quantization method for learning binary compact codes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2938–2945, 2013.
J.-P. Heo, Y. Lee, J. He, S.-F. Chang, and S.-E. Yoon. Spherical hashing. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2957–2964. IEEE, 2012.
J. Wang, S. Kumar, and S.-F. Chang. Semi-supervised hashing for scalable image retrieval. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 3424–3431. IEEE, 2010.
Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Advances in neural information processing systems (NIPS), pages 1753–1760, 2009.
Unsupervised hashing methods use hand-crafted features: 2, 3, 29, 39
A. Alahi, R. Ortiz, and P. Vandergheynst. Freak: Fast retina keypoint. In IEEE conference on Computer vision and pattern recognition (CVPR), pages 510–517. Ieee, 2012.
M. Calonder, V. Lepetit, C. Strecha, and P. Fua. Brief: Binary robust independent elementary features. European conference on Computer Vision (ECCV), pages 778–792, 2010.
S. Leutenegger, M. Chli, and R. Y. Siegwart. Brisk: Binary robust invariant scalable keypoints. In IEEE International Conference on Computer Vision (ICCV), pages 2548–2555. IEEE, 2011.
E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. Orb: An efficient alternative to sift or surf. In IEEE international conference on Computer Vision (ICCV), pages 2564–2571. IEEE, 2011.
Image similarity search in big datasets has gained tremendous attentions in different applications such as information retrieval, data mining and pattern recognition.
Hashing functions provide a binary code to each image, and consequently reducing the similarity search, calculating the Hamming distance.
Y. Gong and S. Lazebnik. Iterative quantization: A procrustean approach to learning binary codes. In In Proc. of the IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2011.
Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Advances in neural information processing systems (NIPS), pages 1753–1760, 2009.
W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang. Supervised hashing with kernels. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2074–2081. IEEE, 2012.
W.-J. Li, S. Wang, and W.-C. Kang. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 1711–1717. AAAI Press, 2016.
Hash functions are designed to extract distinctive patterns from images relevant to their semantic categorizes
K. Lin, J. Lu, C.-S. Chen, and J. Zhou. Learning compact binary descriptors with unsupervised deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1183–1192, 2016.
H.-F. Yang, K. Lin, and C.-S. Chen. Supervised learning of semantics-preserving hash via deep convolutional neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
S. Huang, Y. Xiong, Y. Zhang, and J. Wang. Unsupervised triplet hashing for fast image retrieval. arXiv preprint arXiv:1702.08798, 2017.
Abstract: Unsupervised deep learning approach called DeepBit to learn compact binary descriptor for efficient visual object matching. Enforce three criterions on binary codes which are minimal loss quantization, evenly distributed codes and uncorrelated bits.
Reduce the computational complexity, several lightweight binary descriptors have been recently proposed such
M. Calonder, V. Lepetit, C. Strecha, and P. Fua. Brief: Binary robust independent elementary features. In Proc. ECCV, 2010.
E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. Orb: an efficient alternative to sift or surf. In Proc. ICCV, 2011.
S. Leutenegger, M. Chli, and R. Y. Siegwart. Brisk: Binary robust invariant scalable keypoints. In Proc. ICCV, 2011.
A. Alahi, R. Ortiz, and P. Vandergheynst. Freak: Fast retina keypoint. In Proc. CVPR, 2012.
supervised approaches: 3, 9, 38, 39, 41, 50, 53
V. Balntas, L. Tang, and K. Mikolajczyk. Bold-binary online learned descriptor for efficient image matching. In Proc. CVPR, 2015.
B. Fan, Q. Kong, T. Trzcinski, Z. Wang, C. Pan, and P. Fua. Receptive fields selection for binary feature description. IEEE Trans. Image Proc., 23(6):2583–2595, 2014.
C. Strecha, A. M. Bronstein, M. M. Bronstein, and P. Fua. Ldahash: Improved matching with smaller descriptors. IEEE Trans. Pattern Anal. Mach. Intell., 34(1):66–78, 2012.
T. Trzcinski, M. Christoudias, P. Fua, and V. Lepetit. Boosting binary keypoint descriptors. In Proc. CVPR, 2013.
T. Trzcinski and V. Lepetit. Efficient discriminative projections for compact binary descriptors. In Proc. ECCV, 2012.
X. Yang and K.-T. Cheng. Ldb: An ultra-fast feature for scalable augmented reality on mobile devices. In Proc. ISMAR, 2012
S. Zhang, Q. Tian, Q. Huang, W. Gao, and Y. Rui. Usb: ultrashort binary descriptor for fast visual matching and retrieval. IEEE Trans. Image Proc., 23(8):3671–3683, 2014.
ㄸncodes the desired similarity relationships and learns a project matrix to compute discriminative binary features.
Local Difference Binary (LDB) applies Adaboost to select optimal sampling pairs.: 50, 51
X. Yang and K.-T. Cheng. Ldb: An ultra-fast feature for scalable augmented reality on mobile devices. In Proc. ISMAR, 2012
X. Yang and K.-T. Cheng. Local difference binary for ultrafast and distinctive feature description. IEEE Trans. Pattern Anal. Mach. Intell., 36(1):188–194, 2014.
Linear Discriminat Analysis (LDA) is also applied to learn binary descriptors; 14, 38
Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell., 35(12):2916–2929, 2013.
C. Strecha, A. M. Bronstein, M. M. Bronstein, and P. Fua. Ldahash: Improved matching with smaller descriptors. IEEE Trans. Pattern Anal. Mach. Intell., 34(1):66–78, 2012.
Recently proposed BinBoost learns a set of projection matrix using the boosting algorithm. Their success is mainly attributed to pair-wise learning with similarity labels, and is unfavorable for the case when transferring the binary descriptor to a new task.: 39, 40
Unsupervised hashing algorithms learn compact binary descriptors whose distance is correlated to the similarity relationship of the original input data.: 2, 14, 34, 46
A. Andoni and P. Indyk. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In Proc. FOCS, 2006.
Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell., 35(12):2916–2929, 2013.
R. Salakhutdinov and G. E. Hinton. Semantic hashing. Int. J. Approx. Reasoning, 50(7):969–978, 2009.
Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Proc. NIPS, 2008.
Locality Sensitive Hashing (LSH) applies random projections to map original data into a low-dimensional feature space, and then performs a binarization.
Semantic hashing (SH) builds a multi-layers Restricted Boltzmann Machines (RBM) to learn compact binary codes for text and documents.
Spectral hashing (SpeH) generates efficient binary codes by spectral graph partitioning.
Iterative qauntization (ITQ) uses iterative optimization strategy to find projections with minimal binarization loss.
Deep CNN to learn a set of hash functions, but they require pair-wised similarity labels or triplets training data.
R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan. Supervised hashing for image retreieval via image representation learning. In Proc. AAAI, 2014.
H. Lai, Y. Pan, Y. Liu, and S. Yan. Simultaneous feature learning and hash coding with deep neural networks. In Proc. CVPR, 2015.
Constructs hash functions as a latent layer in the deep CNN. but their method belongs to supervised learning.
Builds three layers hierarchical neural networks to learn discriminative projection matrix, but their method does not take the advantage of deep transfer learning, thus makes the binary codes less effective.
Abstract: CNN based unsupervised hashing designed under three principles
CNN-based hashing methods: 4, 5, 6, 7, 8, 9, 10
stacked Restricted Boltzmann Machines (RBMs) to encode binary codes
replace rotation invariance loss in DeepBit
Supervised hashing: 4, 5, 6, 11, 12
hidden layer to learn binary hash code using softmax layer on the top. Kevin Lin, Huei-Fang Yang, Jen-Hao Hsiao, and ChuSong Chen, “Deep learning of binary hash codes for fast image retrieval,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015, pp. 27–35.
adopt a learns binary codes for all training data and learn hash functions basis of the learned codes. Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan, “Supervised hashing for image retrieval via image representation learning.,” in AAAI, 2014, vol. 1, p. 2.
introduce hashing scheme based on stacked RBMs and Siamese Net, stacked RBMs are learn initial parameters and fine tuned through a Siamese Net Jie Lin, Olivier Morere, Vijay Chandrasekhar, Antoine Veillard, and Hanlin Goh, “Deephash: Getting regularization, depth and fine-tuning right,” arXiv preprint arXiv:1501.04711, 2015.
use a triplet loss function to minimize the Hamming distance between neighbor pairs while preserving relative similarity for non-neighbor pairs with relaxed empirical penalty. Viet-Anh Nguyen and Minh N Do, “Deep learning based supervised hashing for efficient image retrieval,” in Multimedia and Expo (ICME), 2016 IEEE International Conference on. IEEE, 2016, pp. 1–6.
divide and encode module to divide intermediate image features into multiple branches, encoded into one hash bit and use triplet loss to finetune. Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan, “Simultaneous feature learning and hash coding with deep neural networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3270–3278.
Semi-Supervised hashing: 7
Unsupervised hashing: 1, 2, 3, 8, 9, 10
hand crafted image features, proposed local sensitive hashing using random projections to construct hash functions. Sariel Har-Peled, Piotr Indyk, and Rajeev Motwani, “Approximate nearest neighbor: Towards removing the curse of dimensionality.,” Theory of computing, vol. 8, no. 1, pp. 321–350, 2012.
hand crafted image features, Iterative Quantization performs PCA and then learns a rotation to minimize the quantization error of mapping transformed data . Yunchao Gong and Svetlana Lazebnik, “Iterative quantization: A procrustean approach to learning binary codes,” in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011, pp. 817–824.
hand crafted image features Yair Weiss, Antonio Torralba, and Rob Fergus, “Spectral hashing,” in Advances in neural information processing systems, 2009, pp. 1753–1760.
semantic hashing, use RBM as auto-encoder to generate binary codes Ruslan Salakhutdinov and Geoffrey Hinton, “Semantic hashing,” International Journal of Approximate Reasoning, vol. 50, no. 7, pp. 969–978, 2009.
Jie Lin, Olivier Morere, Julie Petta, Vijay Chandrasekhar, and Antoine Veillard, “Tiny descriptors for image retrieval with unsupervised triplet hashing,” arXiv preprint arXiv:1511.03055, 2015.
DeepBit to learn nonlinear mapping function by inserting a latent layer, construct pair-wise training data by combining the original images Kevin Lin, Jiwen Lu, Chu-Song Chen, and Jie Zhou, “Learning compact binary descriptors with unsupervised deep neural networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1183–1192.
Deep Learning
Abstract: train CNN coupled with unsupervised discriminative clustering and use the cluster membership as a soft supervision to discover shared attributes from the clusters while maximizing their separability.
Supervised attribute learning methods: 7, 16, 28, 48, require large label.
S. Branson, C. Wah, F. Schroff, B. Babenko, P. Welinder, P. Perona, and S. Belongie. Visual recognition with humans in the loop. In ECCV, 2010.
A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. In CVPR, 2009.
C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009.
Y. Wang and G. Mori. A discriminative latent model of object classes and attributes. In ECCV, 2010.
drawback by mining attributes from image features to reduce inter category confusions: 33, 35, 37
S. Ma, S. Sclaroff, and N. Ikizler-Cinbis. Unsupervised learning of discriminative relative visual attributes. In ECCVW, 2012.
D. Parikh and K. Grauman. Interactively building a discriminative vocabulary of nameable attributes. In CVPR, 2011.
M. Rastegari, A. Farhadi, and D. Forsyth. Attribute discovery via predictable discriminative binary codes. In ECCV, 2012.
Supervised attribute learning: 7, 16, 28, 48
S. Branson, C. Wah, F. Schroff, B. Babenko, P. Welinder, P. Perona, and S. Belongie. Visual recognition with humans in the loop. In ECCV, 2010.
A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. In CVPR, 2009.
C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009.
Y. Wang and G. Mori. A discriminative latent model of object classes and attributes. In ECCV, 2010.
Unsupervised learning 11, 29, 44, 4, 5, 6, 24, 53
C. Doersch, A. Gupta, and A. A. Efros. Mid-level visual element discovery as discriminative mode seeking. In NIPS, 2013.
Y. Li, L. Liu, C. Shen, and A. van den Hengel. Mid-level deep pattern mining. In CVPR, 2015.
S. Singh, A. Gupta, and A. A. Efros. Unsupervised discovery of mid-level discriminative patches. In ECCV, 2012.
L. Bo, X. Ren, and D. Fox. Unsupervised feature learning for RGB-D based object recognition. In ISER, 2012.
L. Bo, X. Ren, and D. Fox. Multipath sparse coding using hierarchical matching pursuit. In CVPR, 2013.
Y.-L. Boureau, N. Le Roux, F. Bach, J. Ponce, and Y. LeCun. Ask the locals: Multi-way local pooling for image recognition. In ICCV, 2011.
K. Y. Hui. Direct modeling of complex invariances for visual object features. In ICML, 2013.
W. Zou, S. Zhu, K. Yu, and A. Y. Ng. Deep learning of invariant features via simulated fixations in video. In NIPS, 2012.
Deep hashing methods: 14, (27, 31, 51, 52, supervised)
V. Erin Liong, J. Lu, G. Wang, P. Moulin, and J. Zhou. Deep hashing for compact binary codes learning. In CVPR, 2015.
H. Lai, Y. Pan, and S. Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR, 2015.
J. Lin, O. Morere, J. Petta, V. Chandrasekhar, and A. Veil- ` lard. Tiny descriptors for image retrieval with unsupervised triplet hashing. arXiv preprint, arXiv:1511.03055v1, 2015.
R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan. Supervised hashing for image retrieval via image representation learning. In AAAI, 2014.
F. Zhao, Y. Huang, L. Wang, and T. Tan. Deep semantic ranking based hashing for multi-label image retrieval. In CVPR, 2015.
Abstract: learn hash functions through Minimum Description Length principle such that learned hash codes maximally compress dataset and used to regenerate inputs. Also develop learning algorithm based on stochastic distributional gradient, avoid notorious difficulty caused by binary output constraints, to jointly optimize parameters of hash function and associated generative model.
Hamming search over binary codes
Semi supervised hashing
optimize some objective function that captures the preferred properties of the hash function in a supervised or unsupervised fashion
Abstract: hypersphere-based hashing function, spherical hashing, to map spatially coherent data points into binary code compared to hyperplane-based hashing functions.
high-dimensional data points hashing techniques: 11, 23, 26
P. Indyk and R. Motwani. Approximate nearest neighbors: toward removing the curse of dimensionality. In STOC, 1998.
A. Torralba, R. Fergus, and Y. Weiss. Small codes and large image databases for recognition. In CVPR, 2008.
Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In NIPS, 2008.
Semi supervised hashing
spectral hashing
iterative quantization
joint optimization
random maximum margin hashing
bag of visual words representation
GIST descriptor
finding nearest neighbor points in high dimensional image descriptor spaces.
tree based methods: 6, 16, 18
J. H. Friedman, J. L. Bentley, and R. A. Finkel. An algorithm for finding best matches in logarithmic expected time. ACM TOMS, 3(3):209–226, 1977.
K. Kim, M. K. Hasan, J.-P. Heo, Y.-W. Tai, and S.-E. Yoon. Probabilistic cost model for nearest neighbor search in image retrieval. Technical report, KAIST, 2012.
D. Nister and H. Stew ´ enius. Scalable recognition with a vo- ´ cabulary tree. In CVPR, 2006.
Binary hashing methods: 11, 13, 25, 7, 10, 15, 22
Y. Gong and S. Lazebnik. Iterative quantization: a procrustean approach to learning binary codes. In CVPR, 2011.
J. He, R. Radhakrishnan, S.-F. Chang, and C. Bauer. Compact hashing with joint optimization of search accuracy and time. In CVPR, 2011.
P. Indyk and R. Motwani. Approximate nearest neighbors: toward removing the curse of dimensionality. In STOC, 1998.
P. Jain, B. Kulis, and K. Grauman. Fast image search for learned metrics. In CVPR, 2008.
A. Joly and O. Buisson. Random maximum margin hashing. In CVPR, 2011.
K. Terasawa and Y. Tanaka. Spherical lsh for approximate nearest neighbor search on unit hypersphere. In Algorithms and Data Structures, volume 4619, pages 27–38, 2007.
J. Wang, S. Kumar, and S.-F. Chang. Semi-supervised hashing for scalable image retrieval. In CVPR, 2010.
Distance based Indexing Methods: 5, 12, 24
R. F. S. Filho, A. Traina, C. J. Traina., and C. Faloutsos. Similarity search without tears: the omni-family of all-purpose access methods. In Int’l Conf. on Data Engineering, 2001.
H. V. Jagadish, B. C. Ooi, K.-L. Tan, C. Yu, and R. Zhang. idistance: An adaptive b +-tree based indexing method for nearest neighbor search. ACM T. on Database Systems, 2005.
J. Venkateswaran, D. Lachwani, T. Kahveci, and C. Jermaine. Reference-based indexing of sequence databases. In VLDB, 2006.
Abstract: Semantic Structurebased unsupervised Deep Hashing (SSDH), construct semantic structure by considering points with distances obviously smaller than others as semantically dissimilar.
Local Sensitive Hashing
learn hash functions from data distributions, usually perform well with shorter binary codes
learning hash codes under supervised settings
Unsupervised hashing methods
Deep learning
Dizaji, K. G., Zheng, F., Nourabadi, N. S., Yang, Y., Deng, C., & Huang, H. (2018). Unsupervised Deep Generative Adversarial Hashing Network. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 3664–3673. link
Yang, E., Deng, C., Liu, T., Liu, W., & Tao, D. (2018). Semantic structure-based unsupervised deep hashing. IJCAI International Joint Conference on Artificial. link
Lin, K., Lu, J., Chen, C.-S., & Zhou, J. (2016). Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks. 1183–1192. link
Huang, S., Xiong, Y., Zhang, Y., & Wang, J. (2017). Unsupervised Triplet Hashing for Fast Image Retrieval. link
Huang, C., Loy, C. C., & Tang, X. (2016). Unsupervised Learning of Discriminative Attributes and Visual Representations. 5175–5184. link
Intelligence, 2018-July, 1064–1070.
Heo, J. P., Lee, Y., He, J., Chang, S. F., & Yoon, S. E. (2012). Spherical hashing. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2957–2964. link
Dai, B., Guo, R., Kumar, S., He, N., & Song, L. (2017). Stochastic Generative Hashing. 1–21. link