mahmoodlab / CONCH

A vision-language foundation model for computational pathology - Nature Medicine
Other
281 stars 22 forks source link

MI-Zero for Slide-level Downstream Tasks #14

Closed Dootmaan closed 2 months ago

Dootmaan commented 2 months ago

Thank you for your great work. I noticed that MI-Zero is used for conducting slide-level zero-shot downstream tasks. I wonder does it mean that CONCH needs to be finetuned again with MI-Zero on slide-level image-text pairs or we can just use the kMaxpool+Avg pool aggregation directly based on the released CONCH weights?

fedshyvana commented 2 months ago

You can directly use the released model! Although just note that the MI-Zero relies on different classes to be mutually exclusive at the ROI-level to work, so it's most appropriate for subtyping tasks (we talk about this briefly in the discussion.)