5g4s / paper

0 stars 0 forks source link

Vision GNN: An Image is Worth Graph of Nodes #35

Open 5g4s opened 1 year ago

5g4s commented 1 year ago

https://arxiv.org/abs/2206.00272

5g4s commented 1 year ago

The widely-used convolutional neural network and transformer treat the image as a grid or sequence structure, which is not flexible to capture irregular and complex objects.

5g4s commented 1 year ago

In this paper, we propose to represent the image as a graph structure and introduce a new Vision GNN (ViG) architecture to extract graphlevel feature for visual tasks.

5g4s commented 1 year ago

We first split the image to a number of patches which are viewed as nodes, and construct a graph by connecting the nearest neighbors.

5g4s commented 1 year ago

Problem

Since the objects are usually not quadrate whose shape is irregular, the commonly-used grid or sequence structures in previous networks like ResNet and ViT are redundant and inflexible to process them.

The over-smoothing phenomenon in deep GCNs will decrease the distinctiveness of node features and lead to performance degradation for visual recognition.

5g4s commented 1 year ago

Approach

image

We introduce more feature transformations and nonlinear activations in our ViG block. We apply a linear layer before and after the graph convolution to project the node features into the same domain and increase the feature diversity. A nonlinear activation function is inserted after graph convolution to avoid layer collapse.

image

5g4s commented 1 year ago

Advantage of graph representation of the image

The graph is more flexible than grid or sequence to model the complex object as an object in the image is usually not quadrate whose shape is irregular