Is it possible for one anchor point to match multiple gts ?

[Icecream-blue-sky]

In Hierarchical Label Assignment, positive labels are assigned to feature points with top k RFD scores with a certain gt. So is it possible for one anchor point to match multiple gts ?

[Chasel-Tsui]

Hi, very insightful question! Actually, one feature point will not be assigned to match multiple gts. In anchor-based detectors, when using the RFLA, those occluded positive samples, which may be assigned to multiple gts, will be randomly assigned to one of the gts. In anchor-free detectors like FCOS, those occluded positive samples will be assigned to the smaller gt. That also means the RFLA cannot handle very densely arranged objects.

[Icecream-blue-sky]

Hi, very insightful question! Actually, one feature point will not be assigned to match multiple gts. In anchor-based detectors, when using the RFLA, those occluded positive samples, which may be assigned to multiple gts, will be randomly assigned to one of the gts. In anchor-free detectors like FCOS, those occluded positive samples will be assigned to the smaller gt. That also means the RFLA cannot handle very densely arranged objects.

I find that even in the FCOS, when using the RFLA, those occluded positive samples, which may be assigned to multiple gts, will be randomly assigned to one of the gts. Below is the code from mmdet/core/bbox/assigners/hierarchical_assigner.py:

def assign_wrt_ranking(self,  overlaps, k, gt_labels=None):
        num_gts, num_bboxes = overlaps.size(0), overlaps.size(1)

        # 1. assign -1 by default
        assigned_gt_inds = overlaps.new_full((num_bboxes,),
                                             -1,
                                             dtype=torch.long)

        if num_gts == 0 or num_bboxes == 0:
            # No ground truth or boxes, return empty assignment
            max_overlaps = overlaps.new_zeros((num_bboxes,))
            if num_gts == 0:
                # No truth, assign everything to background
                assigned_gt_inds[:] = 0
            if gt_labels is None:
                assigned_labels = None
            else:
                assigned_labels = overlaps.new_full((num_bboxes,),
                                                    -1,
                                                    dtype=torch.long)
            return assigned_gt_inds

        # for each anchor, which gt best overlaps with it
        # for each anchor, the max iou of all gts
        max_overlaps, argmax_overlaps = overlaps.max(dim=0)
        # for each gt, topk anchors
        # for each gt, the topk of all proposals
        gt_max_overlaps, gt_argmax_overlaps = overlaps.topk(k, dim=1, largest=True, sorted=True)

        assigned_gt_inds[(max_overlaps >= 0)
                             & (max_overlaps < 0.8)] = 0
        #assign wrt ranking
        # TODO: occlued pos samples will be sent to later gt not smaller gt ? 
        for i in range(num_gts):
            for j in range(k):
                max_overlap_inds = overlaps[i,:] == gt_max_overlaps[i,j]
                assigned_gt_inds[max_overlap_inds] = i + 1

        if gt_labels is not None:
            assigned_labels = assigned_gt_inds.new_full((num_bboxes,), -1)
            pos_inds = torch.nonzero(
                assigned_gt_inds > 0, as_tuple=False).squeeze()
            if pos_inds.numel() > 0:
                assigned_labels[pos_inds] = gt_labels[
                    assigned_gt_inds[pos_inds] - 1]
        else:
            assigned_labels = None

        return assigned_gt_inds

the code will always assign occluded positive samples to the last gt (i.e., the num_gts-th)

for i in range(num_gts):
      for j in range(k):
          max_overlap_inds = overlaps[i,:] == gt_max_overlaps[i,j]
          assigned_gt_inds[max_overlap_inds] = i + 1

So the code in mmdet/models/dense_heads/rfla_fcos_head.py won't make any difference in the results：

  inds_mask = inds[...,None].repeat(1, num_gts) #num_points, num_gts
  point_mask = torch.arange(num_gts).repeat(num_points, 1).cuda() # num_points, num_gts
  assigned_mask = (inds_mask == point_mask)
  areas[assigned_mask == False] = INF
  # for occluded samples
  min_area, min_area_inds = areas.min(dim=1)

  labels = gt_labels[min_area_inds]
  labels[min_area == INF] = self.num_classes  # set as BG
  bbox_targets = bbox_targets[range(num_points), min_area_inds]

I'm not sure if my thinking is correct, please correct me if I'm wrong

[Chasel-Tsui]

I have checked it again, it seems that the assigning is random in this case. Thanks for your reminder, we will fix the bugs or ambiguous statements in the later version.

[Icecream-blue-sky]

I have checked it again, it seems that the assigning is random in this case. Thanks for your reminder, we will fix the bugs or ambiguous statements in the later version. I find another piece of code that doesn't make sense... In mmdet/core/bbox/assigners/hierarchical_assigner.py, the function assign_wrt_ranking:

# 1. assign -1 by default
assigned_gt_inds = overlaps.new_full((num_bboxes,),  -1, dtype=torch.long)
...
assigned_gt_inds[(max_overlaps >= 0) & (max_overlaps < 0.8)] = 0

Since the assigned_gt_inds is set -1 by default, the code 'assigned_gt_inds[(max_overlaps >= 0) & (max_overlaps < 0.8)] = 0' is meaningless which won't change anything.