Sik-Ho Tang | Brief Review -- Representation Learning by Learning to Count.

[NorbertZheng]

Sik-Ho Tang. Brief Review — Representation Learning by Learning to Count.

[NorbertZheng]

Overview

Self-Supervised Learning By Counting Number of Visual Features.

Representation Learning by Learning to Count, Counting, by University of Bern, and University of Maryland. 2017 ICCV, Over 300 Citations. Self-Supervised Learning, Image Classification.

• Self-supervised learning is achieved by counting number of visual features (feature engineering!!!).

[NorbertZheng]

Counting

Conceptual Idea

The number of visual primitives in the whole image should match the sum of the number of visual primitives in each tile.

• **To obtain a supervision signal useful to learn to count, an image is partitioned into non-overlapping regions.

We get more data!!! The number of visual primitives in each region should sum up to the number of primitives in the original image.

• It is hypothesized that the model needs to disentangle the image into high-level factors of variation, such that the complex relation between the original image and its regions is translated to a simple arithmetic operation.

[NorbertZheng]

Arithmetic calculation of high-level factors (i.e. high-level features selected by feature engineering).

[NorbertZheng]

Contrastive Loss

Training AlexNet to learn to count.

Assume $x$ is color image input, the naïve way for training the network, $D$ is downsampling operator, $T$ is tiling operator to divide the image $x$ into 4 non-overlapping parts:

where $\Phi$ is the CNN to be learnt to count the visual features.

• However, this loss has trivial solution of equaling to zero!!!
• To avoid such a scenario, a contrastive loss is used to enforce that the counting feature should be different between two randomly chosen different images.

Therefore, for any $x\neq y$, we would like to minimize:

where the constant scalar $M=10$.

The contrastive term will introduce a tradeoff that will push features towards counting as many primitives as is needed to differentiate images from each other.

[NorbertZheng]

Network Architecture

Input is 114×114 image. AlexNet with ReLU at the end is used.

• Final layer has 1000 outputs to count 1000 visual features.

The counting network is trained on the 1.3M (i.e. 1'300'000) images from the training set of ImageNet.

[NorbertZheng]

A kind of disentangled (?) representation, but selected through feature engineering.

[NorbertZheng]

Results

Evaluation of transfer learning on PASCAL.

The proposed method either outperforms previous methods or achieve the second best performance.

ImageNet classification with a linear classifier.

Places classification with a linear classifier.

The proposed method achieves a performance comparable to the other state-of-the-art methods on the ImageNet dataset and shows a significant improvement on the Places dataset.

[NorbertZheng]

Reference

• [2017 ICCV] [Counting] Representation Learning by Learning to Count.