search

facebookresearch / sam2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
12.43k stars 1.15k forks source link

Video segmentation with SAM 2 with boxes? #97

Open abhamedewar opened 3 months ago

abhamedewar commented 3 months ago

The tutorial explains passing clicks/points to segment objects in the video frames. Is there a way to pass bounding box instead of clicks/points to track objects in the video frames?

sinAshish commented 3 months ago

https://github.com/roboflow/notebooks/blob/main/notebooks/how-to-segment-images-with-sam-2.ipynb

I found this tutorial that could be helpful.

harper-lambda commented 2 months ago

The tutorial explains passing clicks/points to segment objects in the video frames. Is there a way to pass bounding box instead of clicks/points to track objects in the video frames?

I was working on this recently and found a solution. Use the SAM2ImagePredictor class from the sam2_image_predictor.py file to get a mask from the bounding box. Then initialize the SAM2 model with "add_new_mask" instead of "add_new_points". Everything else should work the same!

sinAshish commented 2 months ago

I'll try this. Thanks