Is lmpm semantic segmentation model or

[hyeonmin11]

instance segmentation model?

[henghuiding]

Hi,

Thank you for the question.

It depends:

• The final output mask of LMPM is binary
  • If the given sentence refers to a single object, then LMPM outputs this object's mask, in this case, it can be viewed as instance segmentation.
  • If the given sentence refers to multiple objects, then LMPM outputs the fusion binary mask that covers all the target objects, in this case, it can be viewed as semantic segmentation, but the semantic class is "all the target objects indicated by the given sentence" instead of common semantic classes like "person".
• Before the final mask fusion, i.e., before the upright * in Fig 5 in the paper, LMPM differentiates instances, from this perspective, it can be viewed as instance segmentation.

It is worth noting that MeViS dataset provides instance-level mask annotations, but for each sentence, its corresponding Ground Truth mask is a binary mask whose foreground region is the target object(s).