There are only 127 slides in the provided test data (There are 129 officially)

[Tonyboy999]

Hi @hrzhang1123, thanks for your great work. Official Camelyon16 provided 129 test slides, while the google drive link in this project only has 127 slides. There are no 'test_114', and 'test_124' in your link. I am not sure how this can affect AUC. But for ACC, Supposing there are 115 true positive samples, then 115/129=0.8915, while 115/127=0.9056.

[Treeboy2762]

To my best understanding, the authors have excluded the two slides due to incorrect annotations. Please refer to the supplemental paper at https://openaccess.thecvf.com/content/CVPR2022/supplemental/Zhang_DTFD-MIL_Double-Tier_Feature_CVPR_2022_supplemental.pdf.

[Tonyboy999]

Thank you! @Treeboy2762 I found the notes about the two excluded test WSIs on the first page of supplemental paper. Two slides in the test set are officially recognized as being incorrectly annotated thus are excluded in the experiments.

And I found the official's explanation about the data here

The following files have been intentionally removed from the original data set:
* normal_86: Originally misclassified, renamed to tumor_111.
* test_049: Duplicate slide.

Test set notes:
* test_114: Does not have exhaustive annotations.

test_049 has been removed officially. So there are 129 WSIs, not 130 in the official test set. However, test_114 doesn't have incorrect label. It doesn't have exhaustive annotations, but it has correct slide-level annotation, which means it can be used to evaluate in MIL task. And the officials didn't mention anything wrong about test124.

[Treeboy2762]

You're welcome! I do not have an answer for that; hope you hear from the authors soon.

[weiaicunzai]

Thank you, so the test114 is a tumor class?