search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
28.52k stars 9.29k forks source link

Using Mask Scoring RCNN in mmdet 3.0+ #10978

Open willcray opened 9 months ago

willcray commented 9 months ago

Hello, I was happily using mmdet 2.28.0 and the Mask IoU / Mask Quality scores outputted by the mask_iou_head in the the Mask Scoring RCNN model. I recently upgraded to mmdet 3.0, and now the output of the inference_detector doesn't include Mask IoU scores. The output is a DetDataSample whose pred_instances only contain bbox, scores (for the bounding boxes), labels, and masks. There is no mask scores / mask quality scores / mask iou scores field.

How do I use models that output mask iou scores within the new context of a DetDataSample?

matthost commented 5 months ago

We were using Mask Scoring RCNN in 2.x. Upgrading now. Noticed there are no longer configs for using mask scoring rcnn. No longer supported?

willcray commented 5 months ago

We were using Mask Scoring RCNN in 2.x. Upgrading now. Noticed there are no longer configs for using mask scoring rcnn. No longer supported?

There are configs in mmdet 3.0+ here

matthost commented 5 months ago

Did you end up getting it working in 3.x ?

willcray commented 5 months ago

Did you end up getting it working in 3.x ?

I can use a MaskScoringRCNN model, but I can't access its mask scores.

matthost commented 4 months ago

In training I am seeing an error:

2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/standard_roi_head.py", line 140, in loss
-- | --
  | 2024-02-08T20:45:39.643Z | mask_results = self.mask_loss(x, sampling_results,
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_scoring_roi_head.py", line 141, in mask_loss
  | 2024-02-08T20:45:39.643Z | loss_mask_iou = self.mask_iou_head.loss_and_target(
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_heads/maskiou_head.py", line 142, in loss_and_target
  | 2024-02-08T20:45:39.643Z | mask_iou_targets = self.get_targets(
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_heads/maskiou_head.py", line 193, in get_targets
  | 2024-02-08T20:45:39.643Z | area_ratios = torch.cat(list(area_ratios))
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_heads/maskiou_head.py", line 240, in _get_area_ratio
  | 2024-02-08T20:45:39.644Z | ratio = gt_mask_in_proposal.areas[0] / (
  | 2024-02-08T20:45:39.644Z | IndexError: index 0 is out of bounds for axis 0 with size 0