ljwztc / CLIP-Driven-Universal-Model

[ICCV 2023] CLIP-Driven Universal Model; Rank first in MSD Competition.
Other
565 stars 67 forks source link

How to train a universal model for multiple binary labeled datasets? #74

Closed Itsanewday closed 5 months ago

Itsanewday commented 5 months ago

Thanks for your kindly sharing! I really need you help on downstream task, especially for medical lesion segmentation. Here are my questions: 1) How can i transfer the model to a specific lesion sementation? 2) In clinical practice, we have lots of binary or very few class foreground. Can i train a universal model with these data from scratch?

MrGiovanni commented 5 months ago

Hi @Itsanewday

Thanks for your questions. We have a follow-up study that might help address both of them. https://www.cs.jhu.edu/~alanlab/Pubs23/li2023suprem.pdf

Particularly, we just released the code that enables transferring the model learning from organ segmentation to solving tumor detection/segmentation tasks in the pancreas. Details can be found here: https://github.com/MrGiovanni/SuPreM/tree/main/target_applications/pancreas_tumor_detection

For Q2, I do not suggest training from scratch. Instead, you could consider loading our pre-trained model checkpoint as a starting point.

Best, Zongwei

Itsanewday commented 5 months ago

Thanks a lot!