Closed Icecream-blue-sky closed 1 year ago
Hi, very insightful question! Actually, one feature point will not be assigned to match multiple gts. In anchor-based detectors, when using the RFLA, those occluded positive samples, which may be assigned to multiple gts, will be randomly assigned to one of the gts. In anchor-free detectors like FCOS, those occluded positive samples will be assigned to the smaller gt. That also means the RFLA cannot handle very densely arranged objects.
Hi, very insightful question! Actually, one feature point will not be assigned to match multiple gts. In anchor-based detectors, when using the RFLA, those occluded positive samples, which may be assigned to multiple gts, will be randomly assigned to one of the gts. In anchor-free detectors like FCOS, those occluded positive samples will be assigned to the smaller gt. That also means the RFLA cannot handle very densely arranged objects.
I find that even in the FCOS, when using the RFLA, those occluded positive samples, which may be assigned to multiple gts, will be randomly assigned to one of the gts. Below is the code from mmdet/core/bbox/assigners/hierarchical_assigner.py:
def assign_wrt_ranking(self, overlaps, k, gt_labels=None):
num_gts, num_bboxes = overlaps.size(0), overlaps.size(1)
# 1. assign -1 by default
assigned_gt_inds = overlaps.new_full((num_bboxes,),
-1,
dtype=torch.long)
if num_gts == 0 or num_bboxes == 0:
# No ground truth or boxes, return empty assignment
max_overlaps = overlaps.new_zeros((num_bboxes,))
if num_gts == 0:
# No truth, assign everything to background
assigned_gt_inds[:] = 0
if gt_labels is None:
assigned_labels = None
else:
assigned_labels = overlaps.new_full((num_bboxes,),
-1,
dtype=torch.long)
return assigned_gt_inds
# for each anchor, which gt best overlaps with it
# for each anchor, the max iou of all gts
max_overlaps, argmax_overlaps = overlaps.max(dim=0)
# for each gt, topk anchors
# for each gt, the topk of all proposals
gt_max_overlaps, gt_argmax_overlaps = overlaps.topk(k, dim=1, largest=True, sorted=True)
assigned_gt_inds[(max_overlaps >= 0)
& (max_overlaps < 0.8)] = 0
#assign wrt ranking
# TODO: occlued pos samples will be sent to later gt not smaller gt ?
for i in range(num_gts):
for j in range(k):
max_overlap_inds = overlaps[i,:] == gt_max_overlaps[i,j]
assigned_gt_inds[max_overlap_inds] = i + 1
if gt_labels is not None:
assigned_labels = assigned_gt_inds.new_full((num_bboxes,), -1)
pos_inds = torch.nonzero(
assigned_gt_inds > 0, as_tuple=False).squeeze()
if pos_inds.numel() > 0:
assigned_labels[pos_inds] = gt_labels[
assigned_gt_inds[pos_inds] - 1]
else:
assigned_labels = None
return assigned_gt_inds
the code will always assign occluded positive samples to the last gt (i.e., the num_gts-th)
for i in range(num_gts):
for j in range(k):
max_overlap_inds = overlaps[i,:] == gt_max_overlaps[i,j]
assigned_gt_inds[max_overlap_inds] = i + 1
So the code in mmdet/models/dense_heads/rfla_fcos_head.py won't make any difference in the results:
inds_mask = inds[...,None].repeat(1, num_gts) #num_points, num_gts
point_mask = torch.arange(num_gts).repeat(num_points, 1).cuda() # num_points, num_gts
assigned_mask = (inds_mask == point_mask)
areas[assigned_mask == False] = INF
# for occluded samples
min_area, min_area_inds = areas.min(dim=1)
labels = gt_labels[min_area_inds]
labels[min_area == INF] = self.num_classes # set as BG
bbox_targets = bbox_targets[range(num_points), min_area_inds]
I'm not sure if my thinking is correct, please correct me if I'm wrong
I have checked it again, it seems that the assigning is random in this case. Thanks for your reminder, we will fix the bugs or ambiguous statements in the later version.
I have checked it again, it seems that the assigning is random in this case. Thanks for your reminder, we will fix the bugs or ambiguous statements in the later version. I find another piece of code that doesn't make sense... In mmdet/core/bbox/assigners/hierarchical_assigner.py, the function assign_wrt_ranking:
# 1. assign -1 by default assigned_gt_inds = overlaps.new_full((num_bboxes,), -1, dtype=torch.long) ... assigned_gt_inds[(max_overlaps >= 0) & (max_overlaps < 0.8)] = 0
Since the assigned_gt_inds is set -1 by default, the code 'assigned_gt_inds[(max_overlaps >= 0) & (max_overlaps < 0.8)] = 0' is meaningless which won't change anything.
In Hierarchical Label Assignment, positive labels are assigned to feature points with top k RFD scores with a certain gt. So is it possible for one anchor point to match multiple gts ?