Question about instance queries

Epiphqny / VisTR

[CVPR2021 Oral] End-to-End Video Instance Segmentation with Transformers

https://arxiv.org/abs/2011.14503

Apache License 2.0

739 stars 95 forks source link

Question about instance queries #13

Closed mxin262 closed 3 years ago

mxin262 commented 3 years ago

Thank you for sharing this great work! I have a question about the instance queries. Are instance categories and instance queries arranged in order? In other words, Whether the final output category of the first instance queries in the frame 1 and the first instance queries in the frame 2 is the same ? If they're same，how to do that? (I can't find description of this in the paper) Thank you!

Epiphqny commented 3 years ago

@Kris-cong We didn't constrain the category of the same instance in different frames, but the ground truth of the same instance is the same, therefore it is only learned by supervision. In the Inference section of 4.1 of the paper, we said,"For instances that have been classified to different categories in different frames, we use the most frequently predicted category as the final instance category".