search

andravin / wincnn

Winograd minimal convolution algorithm generator for convolutional neural networks.
Apache License 2.0
600 stars 145 forks source link

Any particular reason to spread denominators into Matrix A B and G? #23

Closed oneengineer closed 5 years ago

oneengineer commented 5 years ago

I see in your README.md, the element in the matrix you give contains denominator. For example in F(6x6,3x3) : G = ⎡ 1 0 0 ⎤ ⎢ ⎥ ⎢-2/9 -2/9 -2/9⎥ ....

A, G and B matrix they all have denominators? Is there any reason to make one matrix large and another matrix small? For example, let's say the first row of G is G1 and first row of BT is BT1 Since they do pairwise product after multiplying a vector, we can make G1' = G1 / x, BT1' = BT1 * x Changing row by x times large/smalleer doesn't affect the result.

So what's the idea behind the way you assign denominators to the three matrices? Is that for precision?

andravin commented 5 years ago

The laplacian interpolation, which is basically used to compute the "inverse" matrix B given matrices A and G, creates some divisions. In signal processing, normally those divisions are moved to matrix G, which represents the filters, because filters are usually constant, and therefore the extra arithmetic is absorbed by the constants.

If the interpolation points are fractions, then matrix A will also will have these fractions (and powers thereof: A is a Vandermonde matrix of the interpolation points). We usually don't use fractional interpolation points until we exhaust the simple values (0, 1, -1, 2, -2), so the fractions in matrix A happen when we use (1/2, -1/2) for F(6,3) or F(4,5).

Anyway winCNN does not do anything smart about moving these fractions around. You might be able to increase numeric accuracy by scaling differently.