distillpub / post--feature-visualization

Feature Visualization
https://distill.pub/2017/feature-visualization/
Creative Commons Attribution 4.0 International
128 stars 33 forks source link

Gram matrix notation unclear - multiplication seems element-wise, but is actually a dot product #15

Closed sliorde closed 6 years ago

sliorde commented 6 years ago

In footnote 3, I think that the notation is confusing. For specific i and j, the element G_{i,j} is a number, equal to the dot product between the (flattened) response of filter i and filter j. But the formula makes it seem as if we do an element-wise multiplication (without summation), between \text{layer}_n\text{[:, :, i]} and \text{layer}_n\text{[:, :, j]} . Such an element-wise multiplication would result in a matrix, and not in a scalar. I know that there is a "dot" in there, but I would also add a summation sign, or at least mention that the operator is a dot product which results in a scalar.

ludwigschubert commented 6 years ago

Sorry for the late response, I used your suggestion almost directly:

image