smilesun
commented
1 year ago Dataset to use:
Acevedo, Matek and MLL.
The task would be to run all the DomainLab methods based on the following cross-validation scheme. train: Acevedo+Matek --> test: MLL train: MLL+Acevedo --> test: Matek train: Matek+MLL --> test: Acevedo
We would need these measures:
accuracy, balanced accuracy, f1-score (macro and micro), AUC (ovo and ova), Mattheus correlation coefficient, and Cohen's Kappa
Ali had previously introduced the datasets to the students.
To compare the algorithms, we also need to do a few baselines:
1) No normalization: running the model with the typical augmentations and see what it achieves 2) Gray scaling: turning every image to grayscale and running the model with typical augmentations 3) Macenko normalization: Similar to the previous ones, first, we need to transform every image using Macenko normalization and then run the model on the datasets 4) Vahadane normalization you can simply use this toolbox for the last two normalization https://staintools.readthedocs.io/en/latest/index.html
5&6) HSV or HED augmentations: to use HSV or HED augmentations in the training process.
The students can use the existing codes for the augmentation parts https://github.com/DIAGNijmegen/pathology-he-auto-augment https://2021.midl.io/slides/full_72_poster.pdf https://digitalslidearchive.github.io/HistomicsTK/index.html https://arxiv.org/pdf/1902.06543.pdf
aliechoes
schoersch
We should have a class called ExpProtocol s.t.
ExpProtocol([Algo1, Algo2, Algo3], Task, StrategyOfTaskVariantion)can generate a plot where the x-coordinate is the degree of domain mixture or a coordinate to quantify different levels of distribution shift generated, while the y-axis is the generalization performance on the hold out domain.