Get feature map filled with zeros in FilterNorm

[QiuJueqin]

Hi, in build_channel_branch function, FilterNorm with filter_type='channel' is a subsequent module after AdaptiveAvgPool2d, which is expected to get a N*C*1*1 feature map as input. But in FilterNorm, you calculate mean in the 2nd dim, which will return the feature map itself because this dim is a singleton dim.

https://github.com/theFoxofSky/ddfnet/blob/b3be7fa2d77939b6beae9f49ae5a48b41169dba5/ddf/ddf.py#L158-L170

So line 162 will always return a zero feature map, and line 163 return an NaN map.

Is it a potential bug?

[theFoxofSky]

not N C 1 1 but N C k k

FilterNorm is not used after Adaptivepool, but applied after the SE structure.

[QiuJueqin]

not N C 1 1 but N C k k

FilterNorm is not used after Adaptivepool, but applied after the SE structure.

It is used after AdaptiveAvgPool2d:

https://github.com/theFoxofSky/ddfnet/blob/b3be7fa2d77939b6beae9f49ae5a48b41169dba5/ddf/ddf.py#L191-L200

[QiuJueqin]

Please reopen the issue if you don't mind

[theFoxofSky]

As the code you mentioned, Filter Norm is applied after the "nn.Conv2d(mid_channels, in_channels * kernel_size ** 2, 1)", instead of "nn.AdaptiveAvgPool2d((1, 1))".

So, FilterNorm is not used after Adaptivepool2d, but applied after the SE structure. Since there is the SE structure, the input size will be N C k * k

[QiuJueqin]

No matter what feature map get into channel_branch, AdaptiveAvgPool2d (line 196) spatially squeezed it into 1*1, and Conv2d with kernel=1 (line 197), ReLU (line 198), and Conv2d with kernel=1 (line 199) did not change the spatial size. So the input feature map to FilterNorm in line 200 will always be 1*1.

[theFoxofSky]

Oh, I got your point. Yes, strictly speaking, the feature maps is 1 1 in spatial. However, its total size is (N, (C k k), 1, 1). So, if you reshape it in the FilterNorm, it becomes N, C, k k (strictly speaking). Anyway, the number of elements is N C k * k

[QiuJueqin]

OK, now I got why I encoutered this error.

When instantiated a DDFPack module, I passed kernel_size=1, which make (N, C, k, k) became (N, C, 1, 1) and raised that bug.

https://github.com/theFoxofSky/ddfnet/blob/b3be7fa2d77939b6beae9f49ae5a48b41169dba5/ddf/ddf.py#L203-L217

I suggest to add an assertion before calling build_channel_branch to ensure in this branch the kernel size ≠ 1.

[theFoxofSky]

Thanks for the advice, I have updated it