search

hkchengrex / MiVOS

[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion. Semi-supervised VOS as well!
https://hkchengrex.com/MiVOS/
MIT License
469 stars 64 forks source link

Propagation failed #48

Closed AppleMax1992 closed 1 year ago

AppleMax1992 commented 1 year ago

Dear author, it is a great work. However, when I use the interaction_gui for mask propagation after I manually scribble the mask for some frames and then click the propagation button, why did the mask propagate not generate a mask for frame normally?

hkchengrex commented 1 year ago

Label a frame -> propagate. You cannot propagate from multiple frames at once.

AppleMax1992 commented 1 year ago

Thank you for your attention. I have followed your suggestion and I only label the first frame of my test video using the "free scribble", then I click the propagation button. The program can run smoothly and there is no error in the log. However, the propagation mask didn't generate automatically for the following frames. Is there anything I did wrong?

AppleMax1992 commented 1 year ago

Here is the labeled first frame, and the propagation is processing. image But when the propagation finished. the second frame can't have the generated mask as shown in the following image. image And this happens on all subsequent frames, none of them are masking automatically.

hkchengrex commented 1 year ago

Hmm... that should not happen. Does it work when you use other types of interactions?

P.S. This project is a bit old and most people use this now: https://github.com/hkchengrex/XMem

AppleMax1992 commented 1 year ago

Yes, Xmem is a better project, faster, and less GPU intensive. But I have the same problem with the Xmem project. I labeled two objects and run the forward propagation. image But yet, there is no mask generated for the subsequent frames. image So I went back to your previous version as MIVOS, but my problem still exists. However, I haven't tried other types of interaction, let me try them and give you feedback, thanks

hkchengrex commented 1 year ago

It might be an instantiation of https://github.com/hkchengrex/XMem/issues/41, if XMem also doesn't work.

AppleMax1992 commented 1 year ago

Great answer, I think that should be the problem, let me first.

AppleMax1992 commented 1 year ago

I reinstall PyTorch 1.11 and CUDA 10.2. It finally works smoothly. Then this issue should be closed. Many thanks for your attention and suggestions.

hkchengrex commented 1 year ago

Glad it worked. It still seems peculiar to me that this would happen...