Weird results when use Segmentation Pass for inference

[luuil]

https://github.com/PeterL1n/RobustVideoMatting/blob/f8a26e27198a93a94bfd06e96b8d5a34d0660f81/inference.py#L127

I changed this line to use Segmentation Pass. (use the pretrained weights rvm_mobilenetv3.pth)

pha, *rec = model(src, *rec, segmentation_pass=True)
fgr = src * pha

But I got weird mask results, something like this, why?

[PeterL1n]

Try pha.sigmoid(), aka fgr = src * pha.sigmoid(). Because the segmentation pass outputs logits. You need a sigmoid to convert it to 0-1 range.

[luuil]

Thank you for your prompt reply. You are right, now the result is correct.

[FengMu1995]

is it necessary to change "pha.sigmoid()" for the train module """ pred_seg = self.model_ddp(true_img, segmentation_pass=True)[0] loss = segmentation_loss(pred_seg, true_seg) """

[vacing]

is it necessary to change "pha.sigmoid()" for the train module

it seems not necessary

https://zhang-yang.medium.com/how-is-pytorchs-binary-cross-entropy-with-logits-function-related-to-sigmoid-and-d3bd8fb080e7