Closed AjeyPaiK closed 1 year ago
@jonasteuwen @EricMarcus-ai
I think we should correct the ROIs when we measure the Dice score to avoid mistakes in metric evaluation. The regions which are unannotated may introduce ambiguity. Attaching screenshots of training curves for two experiments (ignore_index=0 in CE loss and corrected ROI in loss + metric evaluation).
So, after reviewing, I don't think this is the correct solution 'unmapped' pixels are actually mapped. They are background (i.e. not annotated). That inspecting the annotation gives you a different conclusion only means we need to annotate more carefully. Can't you achieve what you try to achieve by ignoring the background in the losses?
Actually, @jonasteuwen the training logs above show exactly that. The performance indicated by grey curves is by the code in this PR. The performance indicated by blue lines is by ignore_index=0
Agreed to setignore_index=0
during training
Fixes {#11}
Following from the issue mentioned above, I have corrected how the loss and metrics are calculated. See below for a visual explanation.
This correction has led to Dice score for the background behaving as expected. After the ROI correction, both the intersection and cardinality are 0 during Dice metric computation. Whenever there is a
nan
in the dice, we replace it with1.0
in ahcore. See here.