DingXiaoH / RepVGG

RepVGG: Making VGG-style ConvNets Great Again
MIT License
3.34k stars 435 forks source link

How is 1x1 conv with identity matrix as kernel equal to Identity layer #97

Closed prakashjayy closed 2 years ago

prakashjayy commented 2 years ago

in the paper it says

This transformation also applies to the identity branch
because an identity can be viewed as a 1 × 1 conv with an
identity matrix as the kernel. 

Can someone help me with the math?

prakashjayy commented 2 years ago

This is a typo in the paper I guess. this should be 3x3 conv with padding. below is the python implementation

import torch
import torch.nn as nn 

conv3 = nn.Conv2d(24, 24, kernel_size=(3,3), stride=(1, 1), padding=1, bias=False)
conv3.weight.data.fill_(0)
for i in range(conv3.weight.data.shape[0]):
    conv3.weight.data[i, i, 1, 1] = 1

x = torch.randn((3, 24, 10, 10))

with torch.no_grad():
    y = conv3(x)
    print(y.shape)
    print(torch.allclose(y, x))