input and weight transform in winograd 4x3

[llxingkongxia]

In file jit_avx512_core_f32_wino_conv_4x3.cpp. input_transform_data(), I don't understand why G_size = 9, The values of G are as follows:

float G[] = {-2.25f, -0.390625f, 0.87890625f, -2.640625f, 0.625f, -0.625f,
           1.5f, -1.5f, -2.640625f};

not

float G[] = { 4.0f, 0.0f, -5.0f, 0.0f, 1.0f, 0.0f ,
                   0.0f,-4.0f, -4.0f, 1.0f, 1.0f, 0.0f,
                   0.0f, 4.0f, -4.0f,-1.0f, 1.0f, 0.0f,
                   0.0f,-2.0f, -1.0f, 2.0f, 1.0f, 0.0f,
                   0.0f, 2.0f, -1.0f,-2.0f, 1.0f, 0.0f,
                   0.0f, 4.0f,  0.0f,-5.0f, 0.0f, 1.0f };

Similarly, in weight_transform_data, G is

float G[] = {0.26890756302521f, 0.688403361344538f, 0.119514472455649f,
            1.13777777777778f, 0.430252100840336f, 0.179271708683473f};

instead of

float G[] = {1.0f/4,     0.0f,    0.0f,
                 -1.0f/6,  -1.0f/6, -1.0f/6,
                 -1.0f/6,   1.0f/6, -1.0f/6,
                  1.0f/24,  1.0f/12,  1.0f/6,
                 1.0f/24, -1.0f/12,  1.0f/6,
                 0.0f,     0.0f,    1.0f};

[densamoilov]

@kwiersch, can you please help with this question?

[kwiersch]

@llxingkongxia I am not exactly sure how these values are obtained, but I believe that when used in the transform kernels (see here for weight and here for input transform kernels), they have the same effect but with fewer operations.

[llxingkongxia]

maybe toom-cook algorighm, G, B, A can be transform to new matrix.

[llxingkongxia]

https://github.com/oneapi-src/oneDNN/commit/5da3e356, The parameters are used to improve numerical accuracy. but these parameters don't look the best. In paper, polynomial interpolation points = {0,-1,1,1/2,-2} is the best for F(4,3). In oneDNN code, polynomial interpolation points = {0,-0.625,0.625,1.5,-1.5}, we can get matrix A B G from tool, and get the parameters as follow: The same with oneDNN. BT = float G[] = {-2.25f, -0.390625f, 0.87890625f, -2.640625f, 0.625f, -0.625f, 1.5f, -1.5f, -2.640625f}; //input transform AT = float G[] = {0.625f, 1.5f, 0.390625f, 2.25f, 0.244140625f, 3.375f}; //output transform G = float G[] = {0.26890756302521f, 0.688403361344538f, 0.119514472455649f, 1.13777777777778f, 0.430252100840336f, 0.179271708683473f}; //weight transform

[kwiersch]

@mgouicem Maybe you can shed some light on the above. Do you know why the Winograd transform parameters were changed in 5da3e35? The commit message says "improve numerical accuracy" but it would be nice to know how that was determined.

[mgouicem]

@llxingkongxia got it right! We used the script provided by the original authors of the Winograd convolution algorithm to get transform matrices with better condition number. Though I don't remember what was the reasoning behind choosing those specific points, sorry.