search

JerryX1110 / RPCMVOS

[AAAI22 Oral] Reliable Propagation-Correction Modulation for Video Object Segmentation
MIT License
78 stars 11 forks source link

about the reweighting operation in two modulators #5

Closed ruoqi77 closed 2 years ago

ruoqi77 commented 2 years ago

Thanks for your excellent work ! I have a detailed problem about the channel reweighting in two modulators. In fig.2, I see both w_p and w_c are sent to both of the two modulators, but in your statement in "Modulator block" part, the reweighting operation is performed separately by the two vectors (i.e, w_p for propagation, w_c for correction). I am confused about this point.

JerryX1110 commented 2 years ago

Thanks for your attention!

As is stated in the paper, w_p is for propagation and w_c for correction, which is named according to the reliable degree of info they contain.

From the perspective of the feature construction view, they are object-level contextual features. And we use the concatenation of weights [w_p;w_c] for both two modulators and the memory encoder in our model so as to introduce object-level cues from them.

Empirically, only using weight w_p or w_c will lose one type of object-level guidance for modulation in the two modulators, thus leading to a slight drop in performance.