mahmoodlab / UNI

Towards a general-purpose foundation model for computational pathology - Nature Medicine
Other
297 stars 38 forks source link

Clarifying augmentations used for student teacher, regarding fine cell detail #9

Open maxmal1 opened 5 months ago

maxmal1 commented 5 months ago

Thank you for releasing the weights attributed to UNI, as well as the fantastic paper. I am having some trouble understanding the use of some of the augmentations used by UNI. This video explaining DINO video timestamp 20:00 mins, suggests that the goal of using a local and global crop for S and T is that the student must learn a more global representation from what is offered, such that small structures shouldn't contribute in favor of a global structure representation. However, in pathology, the small structures matter a lot, like cell nuclei etc. Is this understanding correct? If so, is it necessary to bypass this for a good representation?