fine-tune the model so that it could segment blood vessels

[WusterHappy]

I found that there were not many medical images in SAM's training data set, such as retinal blood vessel images, so I wanted to fine-tune the model so that it could segment blood vessels. I used the DRIVE data set, input the result of coarse segmentation using Unet network into the prompt_encoder, and then fine-tune the predicted value generated by SAM and calculate the ground loss. I don't know whether this is feasible or what are the loopholes in my thinking? I would appreciate your reply

[kampelmuehler]

5

[JunMa11]

@WusterHappy

We provide a step-by-step tutorial on fine-tuning SAM on 2D and 3D medical image datasets. It requires less than 10G GPU memory. Hope that it could be useful.

https://github.com/bowang-lab/MedSAM#model-training-video-tutorial

[aadilmehdis]

@WusterHappy @kampelmuehler @JunMa11

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

---

Updates:

• Most of the code used were from open-source repos already available earlier in this thread! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.

• Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

  📢 Update Note:

• Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

[aadilmehdis]

@WusterHappy @kampelmuehler @JunMa11

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

Updates:

• Most of the code used were from open-source repos already available earlier in this thread! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.
• Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

📢 Update Note:

• Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

📢 Update Note: @WusterHappy @kampelmuehler @JunMa11

Hello everyone! 👋

I've scaled up the platform to handle a larger number of users and made sure that the platform is robust enough for a smoother experience. I've also added a feature that sends you an email notification when a job has started and when it has finished fine-tuning. 🔔 It might take a couple of minutes for the email to come through, so please hang tight!

Furthermore, in order to manage GPU load more efficiently, I am currently resizing all images to a uniform 256x256 resolution. I am working to support varied dimensions and will incorporate this feature soon.

I'd really appreciate it if you could take it for a spin! Any feedback or issues, big or small, can be addressed directly to me at aadilmehdi99@gmail.com. Your input is extremely invaluable as I continue to improve the platform!

Thanks