Vision GNN: An Image is Worth Graph of Nodes

[5g4s]

https://arxiv.org/abs/2206.00272

[5g4s]

The widely-used convolutional neural network and transformer treat the image as a grid or sequence structure, which is not flexible to capture irregular and complex objects.

[5g4s]

In this paper, we propose to represent the image as a graph structure and introduce a new Vision GNN (ViG) architecture to extract graphlevel feature for visual tasks.

[5g4s]

We first split the image to a number of patches which are viewed as nodes, and construct a graph by connecting the nearest neighbors.

[5g4s]

Problem

Since the objects are usually not quadrate whose shape is irregular, the commonly-used grid or sequence structures in previous networks like ResNet and ViT are redundant and inflexible to process them.

The over-smoothing phenomenon in deep GCNs will decrease the distinctiveness of node features and lead to performance degradation for visual recognition.

[5g4s]

Approach

We introduce more feature transformations and nonlinear activations in our ViG block. We apply a linear layer before and after the graph convolution to project the node features into the same domain and increase the feature diversity. A nonlinear activation function is inserted after graph convolution to avoid layer collapse.

[5g4s]

Advantage of graph representation of the image

The graph is more flexible than grid or sequence to model the complex object as an object in the image is usually not quadrate whose shape is irregular