Train the SAM directly

[TK-31]

I tried to train the sam model with checkpoint directly, but it had no change after loss.backward() though I use sam.train() before. Do I need other configuration?

[luca-medeiros]

I made a finetune repo for sam for coco datasets https://github.com/luca-medeiros/lightning-sam

[aadilmehdis]

@TK-31 @luca-medeiros

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

---

Updates:

• Most of the code used were from open-source repos already available earlier in this thread, (@luca-medeiros) ! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.

• Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

  📢 Update Note:

• Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

[luca-medeiros]

@aadilmehdis Is it closed source?

[aadilmehdis]

@TK-31 @luca-medeiros

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

Updates:

• Most of the code used were from open-source repos already available earlier in this thread, (@luca-medeiros) ! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.
• Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

📢 Update Note:

• Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

📢 Update Note: @TK-31 @luca-medeiros

Hello everyone! 👋

I've scaled up the platform to handle a larger number of users and made sure that the platform is robust enough for a smoother experience. I've also added a feature that sends you an email notification when a job has started and when it has finished fine-tuning. 🔔 It might take a couple of minutes for the email to come through, so please hang tight!

Furthermore, in order to manage GPU load more efficiently, I am currently resizing all images to a uniform 256x256 resolution. I am working to support varied dimensions and will incorporate this feature soon.

I'd really appreciate it if you could take it for a spin! Any feedback or issues, big or small, can be addressed directly to me at aadilmehdi99@gmail.com. Your input is extremely invaluable as I continue to improve the platform!

Thanks

[AnnemSony]

I tried to train the sam model with checkpoint directly, but it had no change after loss.backward() though I use sam.train() before. Do I need other configuration?

Can you help me how to train the sam model on my own dataset.