Is it possible to replace the way computing soft weights with simple conv?

[Kimilovesy]

Hi,

First, thanks for sharing the code, get a lot of inspirations. While I am wondering whether it is possible to replace the way you compute the weights that feed into the VLAD core in the original paper with simple tf.conv2(...)? As shown in the pic, actually, the size of the filters are 1 X 1 X Dim X # of Clusters. It would be nice if you can check whether my understanding is correct. Thanks in advance!

[sitzikbs]

Im not sure I understand the question. What about the centers ?

[Kimilovesy]

In my opinion, the centers are trainable, and they are in the same dim as the input. So we only need to initialize the centers as [# of clusters, dim] using Xavier method or anything else as you also mentioned initialize with K-means does not yield any performance gain. At each training step, we will get updated clusters, right?

[Kimilovesy]

And my question is whether we can replace this part of code with simple convolution? As shown in the original paper, they use 1x1xDxK filters.

[sitzikbs]

I am a bit rusty on the details (not my paper, just implemented it) but your idea seems ok. Notice however that the weights are only shared across k, so, i think you can try to replace the code. let me know if it comes out the same.

[pdpdpd2013]

Hi,

I am also working on netVLAD for place recognition.

In my opinion, VLAD layer is not just another conv2d layer. So you cannot replace it with tf.conv(). The reason is that VLAD layer accumulates the global residual of all pixels according to Equation (4) in the original paper https://arxiv.org/pdf/1511.07247.pdf

For VLAD layer, each pixel in the output (size WxHx64) contains information from all pixels of the input (size WxHx512). If you use conv2d layer instead, each pixel in the output only contains information from local pixels of the input.

[sitzikbs]

Closing due to lack of activity.