Limitation to Partial_conv3

[liamsun2019]

Hi author，

Based on the implementation of Partial_conv3, it looks that this module only handles the case where input channels equal output channels. Just wonder how it can be applied to regular nn.Conv2d, i.e, input channels are different from output channels. I think that's the common case instead. Thanks for your time.

[JierunChen]

Hi, thanks for your interest in this work. We use a PConv followed by a PWConv (conv 1x1) to approximate a regular conv, e.g., conv 3x3. Therefore, in your case, you can adjust the number of output channels of the PWConv. Note that activations should not be placed between the PConv and the PWConv as it hurts the expressiveness.

[liamsun2019]

Appreciate your quick feedback. According to your comment, for following regular conv3x3:

nn.Conv2d(c1, c2, kernel_size, stride, padding, dilation, groups, bias)

In the cases where kernel_size=3, stride=1, padding=1, groups=1, bias=False, above convolution can be replaced with following sequencial operations:

Partial_conv3(c1, 4, 'split_cat') nn.Conv2d(c1, c2, 1, bias=False) nn.BatchNorm2d(c2) nn.ReLU(inplace=True) nn.Conv2d(c2, c2, 1, bias=False)

Not sure if it follows your meaning. Need your suggestion. Thanks.

[JierunChen]

Appreciate your quick feedback. According to your comment, for following regular conv3x3:

nn.Conv2d(c1, c2, kernel_size, stride, padding, dilation, groups, bias)

In the cases where kernel_size=3, stride=1, padding=1, groups=1, bias=False, above convolution can be replaced with following sequencial operations:

Partial_conv3(c1, 4, 'split_cat') nn.Conv2d(c1, c2, 1, bias=False) nn.BatchNorm2d(c2) nn.ReLU(inplace=True) nn.Conv2d(c2, c2, 1, bias=False)

Not sure if it follows your meaning. Need your suggestion. Thanks.

In your case, nn.Conv2d(c1, c2, kernel_size=3, stride=1, padding=1, bias=False) can be replaced by nn.Sequential(Partial_conv3(c1, n_div=4), nn.Conv2d(c1, c2, kernel_size=1, bias=False)).

Note you may modify the default settings of Partial_conv3, e.g., the kernel_size, as needed.

[liamsun2019]

Big thanks. I will try it out.

[liamsun2019]

Another question. The network parameter number increases greatly after the replacement. What's the root cause for that? Does it impact on inference time?

[JierunChen]

Another question. The network parameter number increases greatly after the replacement. What's the root cause for that? Does it impact on inference time?

Hi, the number of parameters should decrease from $9c_1c_2$ to $(\frac{9}{16}c_1c_1 + c_1c_2)$, if $14.2c_2$ > $c_1$.

[liamsun2019]

Thanks for your comment. The issue could be closed.