search

yeezhu / SPN.pytorch

PyTorch implementation of "Soft Proposal Networks for Weakly Supervised Object Localization", ICCV 2017.
http://yzhu.work/spn.html
MIT License
211 stars 37 forks source link

Transfer matrix definition #24

Closed vadimkantorov closed 6 years ago

vadimkantorov commented 6 years ago

In the paper and the code you define D as: D(ij, pq) = ||u_ij - u_pq|| * exp(-((i - p)**2 + (j - q)**2)/(2 * sigma**2)) where u is the features tensor, and i,j,p,q are coordinates of two feature vectors.

The first term is dissimilarity (when it's large, features are very dissimilar), the second is similarity (when it's large, features are very close geometrically). Please correct me if I'm getting it wrongly.

During update of objectness vector: M_i = \sum_j M_j * D_ji, i.e. M_i will have high objectness if locations j have high objectness and are dissimilar to i.

Does it make sense to mix dissimilarity and similarity?

Thanks!

yeezhu commented 6 years ago

D represents dissimilarity between receptive fields, and the random walker accumulates mass at nodes that have high dissimilarity with other nodes, thus salient object regions get high objectness confidence other than backgrounds. The normalized spatial distance is a kind of weight for feature dissimilarity.

vadimkantorov commented 6 years ago

I kind of understand now. Thanks!

Can you please clarify this line https://github.com/yeezhu/SPN.pytorch/blob/master/spnlib/spn/src/generic/SoftProposalGenerator.cu#L117 (kernel_(UpdateProposal))? If I understand well, it does M = M + DM (instead of M = DM in the paper). Am I right?

yeezhu commented 6 years ago

In line 100 proposalBuffer = DM - M. So it does M = M + DM - M = DM in line 117.

vadimkantorov commented 6 years ago

Got it. Thanks!