zhenzi0322 commented 1 year ago

How to Train Your Dataset.

SBoulanger commented 1 year ago

https://github.com/facebookresearch/segment-anything/issues/5

HIMANSHUSINGHYANIA commented 1 year ago

All rights reserved.

This source code is licensed under the license found in the

LICENSE file in the root directory of this source tree.

import torch from torch import nn from torch.nn import functional as F

from typing import Any, Dict, List, Tuple

from .image_encoder import ImageEncoderViT from .mask_decoder import MaskDecoder from .prompt_encoder import PromptEncoder

class Sam(nn.Module): mask_threshold: float = 0.0 image_format: str = "RGB"

def __init__(
    self,
    image_encoder: ImageEncoderViT,
    prompt_encoder: PromptEncoder,
    mask_decoder: MaskDecoder,
    pixel_mean: List[float] = [123.675, 116.28, 103.53],
    pixel_std: List[float] = [58.395, 57.12, 57.375],
) -> None:
    """
    SAM predicts object masks from an image and input prompts.

aadilmehdis commented 1 year ago

@zhenzi0322 , @SBoulanger , @HIMANSHUSINGHYANIA

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

Updates:

Most of the code used were from open-source repos already available earlier in this thread, (@giswqs)! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.
Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

📢 Update Note:
Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

giswqs commented 1 year ago

@aadilmehdis Is your source code for fine-tuning available somewhere?

aadilmehdis commented 1 year ago

@zhenzi0322 , @SBoulanger , @HIMANSHUSINGHYANIA

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

Updates:

Most of the code used were from open-source repos already available earlier in this thread, (@giswqs)! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.

Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

📢 Update Note:

Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

📢 Update Note: @zhenzi0322 , @SBoulanger , @HIMANSHUSINGHYANIA

Hello everyone! 👋

I've scaled up the platform to handle a larger number of users and made sure that the platform is robust enough for a smoother experience. I've also added a feature that sends you an email notification when a job has started and when it has finished fine-tuning. 🔔 It might take a couple of minutes for the email to come through, so please hang tight!

Furthermore, in order to manage GPU load more efficiently, I am currently resizing all images to a uniform 256x256 resolution. I am working to support varied dimensions and will incorporate this feature soon.

I'd really appreciate it if you could take it for a spin! Any feedback or issues, big or small, can be addressed directly to me at aadilmehdi99@gmail.com. Your input is extremely invaluable as I continue to improve the platform!

Thanks

Jeff1933 commented 1 year ago

@aadilmehdis Excuse me, After uploading the original image, click the green button below the image to upload the corresponding annotated image? But after I do that,nothing is happening. I afraid that I need a excemple to use it,

ramdhan1989 commented 11 months ago

@zhenzi0322 , @SBoulanger , @HIMANSHUSINGHYANIA

Hey guys, 👋

Built a quick app to finetune SAM (https://app.instalabel.ai/) on custom data (for those who don't wanna go through the hassle of changing and debugging the code 😅). The app allows users to directly train a fine-tuned SAM model by following a few simple steps:

Log in or create an account Upload around 15-20 samples from your dataset to the platform Upload individual ground truth segmentation masks for each sample Click 'Train annotator'. This will finetune SAM to the data you sent. May need to wait for 15-20 minutes for it to train. ⏳ Go to the 'Annotation' tab and upload any other images you want to try the finetuned SAM on. You can test it out with a few bounding box prompts. Otherwise, you can save the finetuned '.pth' file yourself to use in your projects by pressing the 'Download Trained Checkpoint' button in the annotation tab! The app is still a little buggy and I'm working to fix them (like refreshing the page after uploading all ground truth masks before you can press the 'Train annotator' button). However, it should still be largely usable! 👍 Hoping this platform can help developers have a simple UI to get finetuned checkpoints for SAM and annotate their data with a fine-tuned Segment Anything Model 🎯

Hope it helps! 🙌

Updates:

Most of the code used were from open-source repos already available earlier in this thread, (@giswqs)! Just whipped up a platform really quick to make it possible for users to directly download fine-tuned SAM checkpoints without having to deal with the hassle of setup.

Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.

📢 Update Note:

Thank you for the positive feedback and interest in our platform! I really appreciate the support. However, I want to address a concern regarding scalability and multiple users. While the platform initially worked well with a few users, the sudden increase in the number of users has exposed some limitations. I am actively working on scaling up the platform to accommodate the more users.

Hope that helps! Feel free to let me know if you guys are facing any other issues with the platform. Can try to address them in time!

Hi can we upload mask labeled? you mentioned this "Upload individual ground truth segmentation masks for each sample" but I only make manual annotation there. Please advise Thank you

aadilmehdis commented 11 months ago

Hi @ramdhan1989,

Previously, uploading a mask was an option, but it led to several inconsistencies. To address this, I've updated the platform to support direct annotation. Before fine-tuning a model, you'll need to manually annotate a small set (5-10 images) from your dataset.

To expedite this initial annotation process, I've introduced AI-assisted annotation tools. Simply draw a bounding box using the AI-assisted tool (marked by a Wand Icon, as shown in the image), and it will automatically generate the polygon annotation.

After your initial annotations are set, click on 'Train Annotator'. Keep an eye on your emails for updates on the training status. Once training is complete, you have two options for annotating the rest of your dataset: use 'Instant Annotate' for quick, automated annotation of the entire dataset (accessible in the Annotate tab, indicated by a blue button above the images), or opt for interactive annotation using the 'Instant Annotate' tool within the annotation interface (icon with a wand with sparkles).

Feel free to reach out if you need further clarification or assistance.

facebookresearch / segment-anything

How to train custom models #180

All rights reserved.

This source code is licensed under the license found in the

LICENSE file in the root directory of this source tree.

Also received a few issues about how users can directly upload their individual ground truth segmentation masks for each sample. I used LabelMe for this. An example of a groundtruth mask is shown here.