search

DearCaat / RRT-MIL

[CVPR 2024] Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology
58 stars 3 forks source link

Some question about the results of other works in the thesis #1

Closed pekingtong closed 4 months ago

pekingtong commented 4 months ago

Nice work, The results of other papers in the article are higher than what they mentioned in their article. Can you provide the seeds of data division or training if possible?

DearCaat commented 4 months ago

Thanks for attention.

For CAMELYON-16, we used 2021, 2022, 2023 three seeds. For other datasets, we used 2021 seed. Note that, instead of validating directly with the official test set, we used three-times 3-fold cross-validation. This is a different setup than the original article (e.g., TransMIL, DTFD-MIL) and may be a reason for the different results.

In addition to that, since CAMELYON-16 has only 398 slides (TCGA-NSCLS ~ 1000 slides), arbitrary random factors or uncontrollable other factors can affect the final results more significantly. Therefore, we did not complete the experiment with the goal of completely reproducing the results of the previous work.

lingxitong commented 4 months ago

hello，for other dataset,you use seed 2021,so the std and mean metrics of other dataset is calculated by different fold?and for camelyon16,the std and mean is cal by seed and fold?is that right?

DearCaat commented 4 months ago

yep, for c16, the std and mean is cal by seed (i.e., (2021 AUC mean of 3 folds + 2022 ... + 2023 ... ) / 3). for other datasets, the std and mean is cal by different folds.