search

z-x-yang / Segment-and-Track-Anything

An open-source project dedicated to tracking and segmenting any objects in videos, either automatically or interactively. The primary algorithms utilized include the Segment Anything Model (SAM) for key-frame segmentation and Associating Objects with Transformers (AOT) for efficient tracking and propagation purposes.
GNU Affero General Public License v3.0
2.77k stars 334 forks source link

New object fails to find #105

Open liruiluo opened 11 months ago

liruiluo commented 11 months ago

Check find_new_objs code, it seems that you think sam's ID for the same object in different frames is the same, but in fact it is different. Is this a code problem? def find_new_objs(self, track_mask, seg_mask): ''' Compare tracked results from AOT with segmented results from SAM. Select objects from background if they are not tracked. Arguments: track_mask: numpy array (h,w) seg_mask: numpy array (h,w) Return: new_obj_mask: numpy array (h,w) ''' new_obj_mask = (track_mask==0) * seg_mask new_obj_ids = np.unique(new_obj_mask) new_obj_ids = new_obj_ids[new_obj_ids!=0]

obj_num = self.get_obj_num() + 1

    obj_num = self.curr_idx
    for idx in new_obj_ids:
        new_obj_area = np.sum(new_obj_mask==idx)
        obj_area = np.sum(seg_mask==idx)
        if new_obj_area/obj_area < self.min_new_obj_iou or new_obj_area < self.min_area\
            or obj_num > self.max_obj_num:
            new_obj_mask[new_obj_mask==idx] = 0
        else:
            new_obj_mask[new_obj_mask==idx] = obj_num
            obj_num += 1
    return new_obj_mask
yamy-cheng commented 10 months ago

The ID of each object is initialized by SAM and maintained by DeAOT. The code is only used to obtain the ID of new objects and is not related to the ID of objects in different frames.

XinHu98 commented 7 months ago

I just find this problem. The initialized obj_num should be self.curr_idx + 1 not self.curr_idx