NielsRogge / Transformers-Tutorials

This repository contains demos I made with the Transformers library by HuggingFace.
MIT License
8.48k stars 1.33k forks source link

Fine-tune error on patch images medical dataset #329

Open sleeping4cat opened 1 year ago

sleeping4cat commented 1 year ago

I was working with this tutorial from huggingface Fine-tune SAM I tried fine-tuning it on 50 images (ground truth + mask) but it was unable to train itself on those images. It gave an too many values to unpack a standard error which I received while working with encode's tutorial and few other tutorials, where it is quite easy to train the decoder of SAM but plotting the images becomes the problematic task.

However, I want to highlight, I also trained SAM on One-shot classification with same images, with one training image, it shows very good quality but when I trained it on all the 50 images, it still gave me one predicted mask per image despite each image having multiple masks. Still it kept on predicting one mask per image.

Right now, I am confused, I don't know where anything is going wrong.

Dataset Structure:

In the notebook for Fine-tune SAM, I had put my dataset folder link in-place of breast-cancer dataset link. and then ran the model

Image Example 2

9

Image Copyright belongs to me and my team