About separable 4d convolution

[shim94kr]

Hello, thank you for introducing nice work and code!

I have a question about separable 4d convolution.

Is the WTA operation explicitly applied to the 4d separable convolution? In the code, I can't see the part of WTA in the conv4d.py. or is the WTA kernel in sec3.2 the same as the other kernel(e.g., spatial kernel)?

Plus, can you explain (or derive) more in details about the way to decompose 4d convolution into the wta and spatial convolution (sec3.2, [4d-convolution -> factorization]? Actually, it was hard to understand how the original 4d convolution be equal to the factorized one by equation.

It would be very appreciated if these questions are answered. Thank you!

[gengshan-y]

Hi, there is no explicit WTA operation in the separable 4D convolution. The 2D convolutions along the (u,v) displacement dimension was referred to as "WTA convolution", which can be specialized to a WTA kernel by learning from data.

The 4D kernel is assumed to be separable (rank-2), and therefore equivalent to the convolution of two 2D kernels. To test how much this assumption holds, there is an experiments in Tab.4 (Full-4D vs Sep-4D).

Let me know if more explanations are needed.

[shim94kr]

Thanks for your quick reply. I still want to clarify whether you denote the kernels having the same form and implementation differently as "spatial" and "wta" kernels.

And can you explain why the separable kernels should be rank-2? Is the rank used for tensor not the matrix?

Thank you!

[gengshan-y]

The WTA and sptial conv are implemented here: https://github.com/gengshan-y/VCN/blob/00c4befdbdf4e42050867996a6f686f52086e01a/models/conv4d.py#L209

The self.conv2 is the WTA conv and the self.conv1 is the spatial conv.

Yes, it's the rank for the 4D tensor. Say if you want a 2D kernel to be separable, its rank has to be one.

[shim94kr]

Though the notion of the rank for the tensor is hard to understand, conceptually I understand what you mean. Thank you so much! I close this issue.