junjiehe96 / FastInst

[CVPR2023] FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation
MIT License
178 stars 16 forks source link

Fix demo mask visualization #32

Closed jneeven closed 8 months ago

jneeven commented 8 months ago

Hi, thanks for the awesome work and making this code available! I love the paper and have been playing around with the demo code a bit just to see how it looks.

This is a draft PR that changes the requirements.txt to prevent package conflicts (and corresponding to https://github.com/junjiehe96/FastInst/blob/main/INSTALL.md#example-conda-environment-setup) so the package can be installed. It also fixes the mask visualization in the demo to include confidence thresholding and to display the label in the right place.

Without these changes, the demo displays all instances regardless of their confidence, and their labels are all displayed in the top left corner of the image.

I've kept this a draft PR as I'm not happy with the solution; this "fix" actually breaks video inference, which is a bug in Detectron itself. The video visualizer pretends to support not having any boxes in the model predictions, but then breaks because it accesses boxes[i] after first setting boxes = None in https://github.com/facebookresearch/detectron2/blob/main/detectron2/utils/video_visualizer.py#L104. Fixing that is beyond the scope of this PR.

I have also only tested this code for instance segmentation, so it won't work for panoptic segmentation. I'll close this PR as I don't intend to work on it any further, but perhaps the code helps someone else.