search

dqshuai / MetaFormer

A PyTorch implementation of "MetaFormer: A Unified Meta Framework for Fine-Grained Recognition". A reference PyTorch implementation of “CoAtNet: Marrying Convolution and Attention for All Data Sizes”
MIT License
210 stars 36 forks source link

I have a question about "linear embbeding" and "non-linear embbeding". #5

Closed manakahasegawa closed 2 years ago

manakahasegawa commented 2 years ago

Thanks for all your great work!

I have two questions about the paper.

  1. Is figure 2 on page 4 of the paper and figure 1 on page 10 of the paper referring to the same architecture?

  2. The term "non-linear embbedding" and "linear embbedding" are used to describe embedding meta-information, but if the figures refer to the same architecture, what is the intention behind the different designations? Neural networks are iterations of processes that perform linear transformations and activation functions that perform nonlinear transformations. Is it correct to say that you used "non-linear embbedding" because you are using an activation function relu that performs a non-linear transformation?

dqshuai commented 2 years ago

hi, it's my bad. All description should be "non-linear embbedding". As you said, we used relu to performs a non-linear transfromation.