search

facebookresearch / segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
46.71k stars 5.54k forks source link

Tutorial: fine-tune SAM on custom dataset (demo notebooks) #355

Open NielsRogge opened 1 year ago

NielsRogge commented 1 year ago

Hi,

SAM was recently added to Hugging Face Transformers 🤗 , and I made some tutorial notebooks to showcase how to fine-tune the model on a different domain (medical images) as well as inference:

https://github.com/NielsRogge/Transformers-Tutorials/tree/master/SAM

yong2khoo-lm commented 1 year ago

Thanks for sharing. Would like to ask, how long does your fine-tune training process take?

load a small dataset of 130 (image, ground truth mask) pairs.

And is the result good?

(I am thinking to fine-tune for detecting objects from satellite/drone images)

NielsRogge commented 1 year ago

how long does your fine-tune training process take?

For me it only took a couple of minutes as my dataset only contains 130 images.

The result is good yes, see also the MedSAM project + paper which fine-tunes SAM on the medical domain, thereby improving performance.

HripsimeS commented 1 year ago

@NielsRogge Hi Niels. I was wondering if there is a way to download your dataset (130 images with masks) on local. load_dataset("nielsr/breast-cancer", split="train") this line loading a dataset, but I would like to download to see the images and masks. Can you please tell me where I can download the ZIP file of breast-cancer dataset. Thanks a lot in advance!

NielsRogge commented 1 year ago

By default, the Datasets library (like Transformers) stores everything in your local cache.

You can always store it using the save_disk method: https://huggingface.co/docs/datasets/process#save.

HripsimeS commented 1 year ago

@NielsRogge thanks for your quick reply. I used save_disk and in my local 3 files are saved

data-00000-of-00001.arrow dataset_info.json state.json

Can you please let me know how I can extract images and masks files using these 3 files. Thanks in advance!

kime541200 commented 1 year ago

@NielsRogge This is truly an amazing project! I have a few questions I would like to ask:

  1. Can I still use the approach provided in this project for fine-tuning if the images in my dataset have different sizes?
  2. Are there any restrictions on the sizes of the images in the dataset?
manimtechrs commented 1 year ago

i had following error while preparing custom dataset for color image .

ValueError: zero-size array to reduction operation minimum which has no identity