A potential problem of warp_feature

[longbowzhang]

Hi @zengwang430521 , I noticed that in your implementation of the warpfeature, a single element in the output transferred F{uv} could potentially correspond to multiple elements in the local feature F{im}. For example, the u,v coordinate of a element in F{uv} is [16,16]. While in the IUV image, the u,v coordinates are of float type. Then, rounding is required to create the correspondence, and thus multiple locations in the IUV image (e.g., an element with [u,v] = [16.1, 16.6] and another element with [u,v] = [16.4, 16.7]). Then two locations in F{im} correspond to the single location in the F{uv}.

Therefore, I am wondering which feature vectors will be transferred?

Best.

[zengwang430521]

Hi, @longbowzhang , You are right, multiple pixels in local feature map may be warpped to the same location in the UV space. In this case, the transferred feature is the mean of these feature vectors.

[longbowzhang]

Hi @zengwang430521, Thanks a lot for your fast reply. Then I have a follow-up question after checking the implementation of warp_feature. https://github.com/zengwang430521/DecoMR/blob/1bcc3bf3b348fe992b720c7fec52e724df7e73a7/models/dense_cnn.py#L19-L66

Maybe I overlook something, but I cannot find the mean operation as you have mentioned

mean of these feature vectors

Therefore, could you elaborate a little bit more on this?

Best.

[zengwang430521]

Hi @longbowzhang , For a location with multiple transferred feature vectors, with _torch.indexadd() function, multiple feature vectors will be added to the same location of _warpedfeature. And _warpedw is used to count the feature vectors added to every location. So with _warped_feature = warped_feature / (warpedw + 1e-8), we get the mean of multiple feature vectors.

I have added more comments for this function. If there is something not clear, feel free to contact us.