search

rmaphoh / RETFound_MAE

RETFound - A foundation model for retinal image
Other
311 stars 63 forks source link

fine-tune in small sample size #20

Open ysqkid opened 6 months ago

ysqkid commented 6 months ago

Hello~Thank you very much for your meaningful work. I wanted to reach out and seek your advice regarding an issue I am facing with overfitting in my model. Due to the limited size of my dataset, which consists of only a few hundred examples, I have opted to use transfer learning for modeling. However, after applying fine-tuning using the provided code, I have encountered overfitting with a test accuracy of only around 0.6.

I would greatly appreciate any suggestions or methods you may have to address overfitting in small datasets during the fine-tuning process.

rmaphoh commented 5 months ago

We have used a few public datasets containing a few hundred images in experiments. No overfitting case was observed. Can you provide more info for your task, e.g. disease, training loss, val loss, and data split. Meanwhile, have you tested the performance using any CNN? Is there overfitting observed?

hasanar1f commented 4 months ago

Sorry for sliding into this thread, May I know the GPU specs or any recommendations for fine-tuning the model?