Hello, your work is very good, but I cannot reproduce your results, and I think there is something wrong with the code of your evaluation indicator.

1.I followed your steps to reproduce and found that the MOTA indicator was greater than 0.9. After careful inspection, I found that it was because you annotated FP in the code, but it was still greater than 0.9 after uncommenting it.

`def mismatches(mappings_prev, mappings_curr, aff_mat, objs, gt_objs): fp = 0 mismat = 0 if mappings_prev == None: return 0

for m in mappings_prev.keys():

for gt, mapped in mappings_curr.items():
    if gt in mappings_prev and gt in gt_objs:
        # skip low confidence objects
        aff_idx_prev = list(mappings_prev.keys()).index(gt)
        aff_idx_curr = list(mappings_curr.keys()).index(gt)
        if aff_mat[0, aff_idx_prev, aff_idx_curr] < 0.5:
            # fp += 1
            continue
        if mapped != mappings_prev[gt]:
            mismat += 1
return mismat, fp

2.I found that in your evaluation index, miss is directly compared with the size between the true value and the detection. This should be incorrect. The correct thing should be to use the true value minus the tracking value to judge the missed detection after the tracking result comes out. Because there are a large number of false detections in your test results, this will result in no missed detections at all during the evaluation. `def eval_tracking(args, objs, gt_objs, mappings_prev, mappings_curr, aff_mat):

matched but affinity < thres, false positive

# gt can't find object, miss
gt_objs_ = dict()
for key, gt_obj in gt_objs.items():
    if gt_obj.size(2) < args.min_obj_points:
        continue
    gt_objs_[key] = gt_obj
gt_objs = gt_objs_

misses = 0
if len(gt_objs) > len(objs):
    misses = len(gt_objs) - len(objs)
mismat, fp = mismatches(mappings_prev, mappings_curr, aff_mat, objs, gt_objs)
if len(gt_objs) == 0:
    return {'na': 1}

3.I gave a modified judgment criterion, but under this judgment criterion there were a large number of missed detections, so that MOTA<0, I don’t know if there is something wrong with the writing.

`def mismatches(mappings_prev, mappings_curr, aff_mat, objs, gt_objs): fp = 0 # 误检 fn = 0 # 漏检 mismat = 0 # 身份切换 tracked = 0 # 成功跟踪到的真实目标数量

if mappings_prev is None:
    # 如果没有前一帧的映射，假设所有真实目标都未被跟踪到，因此都是漏检
    fn = len(gt_objs)
    return mismat, fp, fn

for gt, mapped in mappings_curr.items():
    if gt in mappings_prev and gt in gt_objs:
        aff_idx_prev = list(mappings_prev.keys()).index(gt)
        aff_idx_curr = list(mappings_curr.keys()).index(gt)
        if aff_mat[0, aff_idx_prev, aff_idx_curr] >= 0.25:
            # 置信度高，认为目标被成功跟踪
            tracked += 1
            if mapped != mappings_prev[gt]:
                # 如果映射的目标ID在前后帧不一致，则认为发生了身份切换
                mismat += 1
        else:
            # 置信度低，认为发生了一次误检
            fp += 1

print("gt:",len(gt_objs))
fn = len(gt_objs) - tracked
print("GT:",len(gt_objs),"FP:",fp,"FN:",fn)
# fp = fp+fn
return mismat, fp

LJacksonPan / RaTrack

Can not reproduce the provided result and some bug in eval #2

for m in mappings_prev.keys():

matched but affinity < thres, false positive