hkchengrex / Mask-Propagation

[CVPR 2021] MiVOS - Mask Propagation module. Reproduced STM (and better) with training code :star2:. Semi-supervised video object segmentation evaluation.
https://hkchengrex.github.io/MiVOS/
MIT License
128 stars 22 forks source link

some question about codes #27

Closed longmalongma closed 3 years ago

longmalongma commented 3 years ago

您好,在您的代码中我有个疑问想请教一下,如果我对meige每个帧分别保存两个key特征图和两个value特征图,如下:

image

应对PropagationModel中的self.PNet(Fs[:,0], Ms[:,0])也分别对应两个key特征图和两个value特征图? image 是不是对应在PropagationNetwork的segment中的输入是不是也要对应两个key特征图和两个value特征图? 除外之外,还需要做哪些改进吗? image

hkchengrex commented 3 years ago

Looks like multi-head attention? You would also need to change how the attention work, i.e., MemoryReader.

longmalongma commented 3 years ago

oks like multi-head attention?

Yes, I want to try it. Apart from the need to modify the MemoryReader, there should be no other places to modify, right?