search

bo-miao / MAMP

[ICME 2022] Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagation.
BSD 3-Clause "New" or "Revised" License
32 stars 6 forks source link

Application for grayscale images #5

Closed loopdigga96 closed 2 years ago

loopdigga96 commented 2 years ago

Hello! Thank you for publishing the paper and the code. I was wondering if your approach could be applied to grayscale images?

bo-miao commented 2 years ago

Hello! Thank you for publishing the paper and the code. I was wondering if your approach could be applied to grayscale images?

Hi,

MAMP adopts two channels to extract features for matching and one dropped channel as the pseudo label for pseudo mask propagation.

Yes. You can design your image format to replace the "matching" channels or the "pseudo label" channel. Intuitively, the "pseudo label" channels should have low correlation with "matching" channel to avoid trivial solution, i.e., to prevent the model from predicting missing data rather than learning matches.

loopdigga96 commented 2 years ago

Thanks @bo-miao, if you have such example of decorrelated channels in mind, it would be great to hear.

bo-miao commented 2 years ago

Thanks @bo-miao, if you have such example of decorrelated channels in mind, it would be great to hear.

An example is Lab color space, it has fewer channel-wise correlations compared with RGB color space [1].

[1] Reinhard, E.; Adhikhmin, M.; Gooch, B.; and Shirley, P. 2001. Color transfer between images. IEEE Computer graphics and applications, 21(5): 34–41.

loopdigga96 commented 2 years ago

@bo-miao Okay, but is it valid to represent single channel grayscale image with LAB?

bo-miao commented 2 years ago

@bo-miao Okay, but is it valid to represent single channel grayscale image with LAB?

The data need at least 2 channels (one for matching and one as label). You can try to combine multi-formats together, e.g., one format for matching and another as label. There would be many kinds of combinations.

loopdigga96 commented 2 years ago

As far as I know there is only one way of representing grayscale images. Using single channel for pixel intensity. What other multi-formats are you referring?

bo-miao commented 2 years ago

As far as I know there is only one way of representing grayscale images. Using single channel for pixel intensity. What other multi-formats are you referring?

The formats that I said are different colour spaces, Lab, RGB, etc. Compared to changing colour spaces, I would recommend optimizing the framework since that belongs to strong novelty.