Loading Segment Anything as foundation model

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.

https://detectron2.readthedocs.io/en/latest/

Apache License 2.0

29.36k stars 7.33k forks source link

Loading Segment Anything as foundation model #5053

Open crypdick opened 11 months ago

crypdick commented 11 months ago

🚀 Feature

FAIR's Segment Anything Model (SAM) is an Apache-licensed foundation model for instance segmentation. Support for fine-tuning SAM is one of the most requested features in the downstream repository (see https://github.com/facebookresearch/segment-anything/issues/5).

This would require adding the SAM weights to the detectron2 model zoo and a helper util to extract the image encoder weights (which is a ViT).

xxxming730 commented 10 months ago

@crypdick Hello,I now know how to implement fine-tuning, but I now want to know how to classify the SAM segmentation results for recognition, and how does it implement training and inference.TY!

crypdick commented 10 months ago

Hi @xxxming730, how did you fine-tune SAM with detectron2?

xxxming730 commented 10 months ago

@crypdick Sorry bro, I didn't know that. I didn't use detectron2 for fine-tuning. I'm working on other ways to fine-tune SAM.