search

facebookresearch / segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
46.73k stars 5.54k forks source link

How can I fine tune the dataset annotated with labelme, which only has images and labels? Thank you. #660

Open biyuefeng opened 8 months ago

biyuefeng commented 8 months ago

How can I fine tune the dataset annotated with labelme, which only has images and labels? Thank you.

Dandelion404 commented 8 months ago

The first thing is this model does not have a "Offcial" training or fine-tuning program. Therefore it cannot be easy. However, has been working on this model for a few weeks and finally got through most of stuck, I got some recommendations.

a. basic thoughts and demo for fine-tuning https://encord.com/blog/learn-how-to-fine-tune-the-segment-anything-model-sam/ This guy gives the main idea for fine-tuning with a small demo. However, you may find it get many things to be improved for training.

b. Fine-tuning code by Jaimi-choi https://github.com/jaime-choi/Training-Segment-Anything-Model/tree/main Recommend. I learnt a lot from them during construction of my one. There are only a few strange features in the code. So plz be careful.

There are some other recommendations but I have not tried.

medic SAM https://github.com/bowang-lab/MedSAM#model-training-video-tutorial Lora https://auto.gluon.ai/stable/tutorials/multimodal/image_segmentation/beginner_semantic_seg.html Venus https://github.com/hu-po/ashenvenus