Discussion on metrics used in segmentation evaluation

ivadomed / ms-lesion-agnostic

Deep Learning contrasts "agnostic" tool for MS lesion segmentation in the spinal cord

MIT License

4 stars 0 forks source link

Discussion on metrics used in segmentation evaluation #30

Open plbenveniste opened 4 weeks ago

plbenveniste commented 4 weeks ago

Here are useful ressources to look into:

Our lab metrics definition: https://github.com/ivadomed/MetricsReloaded
Panoptica: Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
Anima metrics: Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis
Other repos: https://github.com/Jingnan-Jia/segmentation_metrics
Another metric computed here: https://github.com/npnl/atlas2_grand_challenge/blob/main/isles/scoring.py#L341
A website for chosing the right metric: https://metrics-reloaded.dkfz.de/

valosekj commented 4 weeks ago

Thanks for opening this issue! @naga-karthik and I originally used the ANIMA package for the SCI lesion segmentation. However, after discovering that ANIMA might not handle some edge cases properly and since ANIMA is written in C++, making it difficult to adapt/extend, we switched to https://github.com/ivadomed/MetricsReloaded. A drawback of the current MetricsReloaded wrapper script is that it is relatively slow when computing multiple metrics across many subjects.