search

DingXiaoH / RepVGG

RepVGG: Making VGG-style ConvNets Great Again
MIT License
3.34k stars 435 forks source link

question: have you ever tried to combine this method with ACNet? #39

Open CoinCheung opened 3 years ago

CoinCheung commented 3 years ago

If we replace the 3x3 conv with the convolution proposed with ACNet, are we expected to have further improvements on the test set ?

DingXiaoH commented 3 years ago

We have not tried that. But we are working to release ACNet v2 (Diverse Branch Block, also accepted by CVPR 2021, https://github.com/DingXiaoH/DiverseBranchBlock). We will try combining them very soon.

CoinCheung commented 3 years ago

@DingXiaoH Thanks for replying !!! I have another question:

We can see from this line: https://github.com/DingXiaoH/RepVGG/blob/a689eaf2c3d8f9d553328a4f7207c8b1a8f20933/repvgg.py#L33 that the identity branch is a nn.BatchNorm2d layer, which should learned a a * x + b mapping of each feature channel.

However, in the fusing process: https://github.com/DingXiaoH/RepVGG/blob/a689eaf2c3d8f9d553328a4f7207c8b1a8f20933/repvgg.py#L84 The values are all set to an constant value of 1.

Why not set these values to the as and bs learned by the bn layer ?

DingXiaoH commented 3 years ago

This kernel_value is a fake "1x1 conv" constructed just for the ease of merging the BN. Because y=x is equivalent to y=conv(x, identity_kernel) There is no such kernel in the block.

CoinCheung commented 3 years ago

@DingXiaoH Thanks for telling this, I have made sense of it. By the ways, what is the arxiv index of acnetv2 ? I cannot find it by searching the title directly in google.

DingXiaoH commented 3 years ago

Working on it. Will be released very soon. https://github.com/DingXiaoH/DiverseBranchBlock

CoinCheung commented 3 years ago

Hi,

I have another question about repvgg, did you use label-smooth/auto-augment or other regularization method to train repvgg_a0-a3, and repvgg_b0-b1 ?