vedaldi / practical-cnn

A VGG practical on convolutional neural networks
154 stars 75 forks source link

Some questions in CNN output formula #1

Open slmjlf opened 9 years ago

slmjlf commented 9 years ago

Hi! Very glad to be the first person starting an issue!

I am a new learner to CNNs and recently I found some problem when reading the MatConvNet documentation or VGG Convolutional Neural Networks Practical.

In Part 1.1 of the practical there is a question "Note that x is indexed by i+i′ and j+j′, but that there is no plus sign between k and k′. Why?" This is also my question. qq 20150317101734

(1) In that formula, why does x has four dimensions but f has only three dimensions? When defining x,y and f we found clearly that x and y are 3-dimensional, f is 4-dimensional. Is there any typo in this formula? (2) Why do the first two dimensions of x are i+i′ and j+j′? Does that mean that (row-1, col-1) of x is not involved in the convolution process? If so, I don't think it is right.

vedaldi commented 9 years ago

Hi, that’s because the convolution applied spatially, but not done across feature channels. This is in principle possible, but it would amount to account for “channel” as a third coordinate analogous to space, which would be odd. For example, there is usually no preferential order in features (although this rule is not always true: there is in local cross-channel normalisation).

On 17 Mar 2015, at 05:11, Zhiyang Wang notifications@github.com wrote:

Hi! Very glad to be the first person starting an issue!

I am a new learner to CNNs and recently I found some problem when reading the MatConvNet documentation or VGG Convolutional Neural Networks Practical.

In Part 1.1 of the practical there is a question "Note that x is indexed by i+i′ and j+j′, but that there is no plus sign between k and k′. Why?" This is also my question.

(1) In that formula, why does $x$ has four dimensions but $f$ has only three dimensions? When defining x,y and f we found clearly that x and y are 3-dimensional, f is 4-dimensional. (2) Why do the first two dimensions of $x$ are i+i′ and j+j′? Does that mean that (row-1, col-1) of x is not involved in the convolution process? If so, I don't think it is right.

— Reply to this email directly or view it on GitHub https://github.com/vedaldi/practical-cnn/issues/1.

slmjlf commented 9 years ago

Thanks for answering, Andrea. I have verified my understandings using my own code and it matches the result in MatConvNet, which means my understandings are right!

What I am arguing is that I think the formula is wrong. It should be like qq 20150319101431 in the VGG practical or qq 20150319101441 in the manual of MatConvNet.

Thanks again for helping!

Zhiyang

vedaldi commented 9 years ago

Hi, except for the bias term b_k and the fact that the indexes have been renamed, is there any difference between the two formulas?

On 19 Mar 2015, at 02:17, Zhiyang Wang notifications@github.com wrote:

Thanks for answering, Andrea. I have verified my understandings using my own code and it matches the result in MatConvNet, which means my understandings are right!

What I am arguing is that I think the formula is wrong. It should be like https://cloud.githubusercontent.com/assets/7233963/6723299/e277b444-ce20-11e4-8ad1-e8161f19da32.jpg in the VGG practical or https://cloud.githubusercontent.com/assets/7233963/6723304/f531aff4-ce20-11e4-8f58-f4b5c5a34219.jpg in the manual of MatConvNet.

Thanks again for helping!

Zhiyang

— Reply to this email directly or view it on GitHub https://github.com/vedaldi/practical-cnn/issues/1#issuecomment-83267848.

slmjlf commented 9 years ago

Hi @vedaldi you didn't understand what I mean!

The above two formulas are surely right and they make no difference. But in the VGG practical and the MatConvNet manual, the two formulas are wrong.

Specifically, in the practical, your formula is qq 20150319223646

However it should be qq 20150319223750

Similarly, in the manual, the formula is qq 20150319224023 The right form should be qq 20150319224034

Hope you can find the differences and understand what I mean.

geumjandi commented 9 years ago

i have a question regarding to this expression that what does i' and j' signifies