1×1 convolutional layer

[Cartus]

So Boyuan propose an unanswered question: if 1×1 convolutional layer can be considered as a fully-connected layer?

The answer is yes based on the description here: https://d2l.ai/chapter_convolutional-neural-networks/channels.html

You can refer to section 6.4.3.

To sum up:

1. The 1×1 convolutional layer is equivalent to the fully-connected layer when applied on a per pixel basis.
2. The 1×1 convolutional layer is typically used to adjust the number of channels between network layers and to control model complexity.

[RaleighZ]

I am still confused by the definition of the fully-connected layer (FCL) over a graph? Let say we have a graph with 128*128 pixels with 3 channels, then we build an FCL over a graph on a per pixel basis. Does it mean the input of this FCL is the value of a single pixel from three channels? And we apply this FCL to every pixel on the graph? If so, it makes sense to say that the 11 kernel can be considered as a fully-connected layer. And in this way, 1\1 kernel serves as a data combinator which combines pixel value from different channels.

[xuuuluuu]

A 1×1 convolution gives you a fully connected linear layer across channels. It normally used to map from many channels to fewer channels.