Why is ARI and VOI topology-aware

[ZefanW]

Any proof? It seems that these are pixel-based metrics that only use 4 basic partitions(TP, FP, TN, FN). And I have not found any related literature before yours claiming that ARI and VOI are related to topology correctness. In fact, ARI is first used as an acronym for Adapted Rand Index in the CREMI (https://cremi.org/) official metric. On their website, they are also mentioned as pixel-based measures.

[HuXiaoling]

Thanks for your question.

Both RI (Rand Index) and VOI (Variation of Information) are essentially measuring the similarity between two data clusterings. They are used as measures of segmentation performances since a segmentation can be regarded as a clustering of pixels. Noting that here, instead of focusing on the binary segmentation map (partitioning the whole image into two classes, foreground or background), we compute the RI/VOI on the integer-valued labeling versions of prediction/GT (each connected component is a separate cluster).

Based on the description above, you might notice that RI and VOI are pixel-based metrics if you only have two clusters, foreground and background classes.

Please let me know if you have more questions.

[1] Jain, Viren, et al. "Boundary learning by optimization with topological constraints." CVPR 2010. [2] Uzunbas, Mustafa Gokhan, Chao Chen, and Dimitris Metaxas. "An efficient conditional random field approach for automatic and interactive neuron segmentation." MedIA 2016.

[ZefanW]

Thanks for your explanation.

Besides the CVPR 2010 paper directly points out that RI is calculated between objects(components) rather than classes, most related research papers seem to omit the details of RI and VOI calculation, which confuses me about how RI and VOI should be used in the foreground/background circumstances.

I also checked the CREMI evaluation source code and both metrics are indeed pixel-based rather than component-based. But the EM neuron segmentation task itself requires all components to be different classes, so a pixel-based metric is equivalent to a component-based one. So I guess it's natural to transfer their implementation to your version. Also, I found a thorough explanation in the Appendix of "Structure-Aware Image Segmentation with Homotopy Warping". That's helpful and I should have seen it earlier.

Thank you again for your helpful response.

[Fivethousand5k]

So, do you mean that we need to attribute different integer values to each connected component with some tool like the Skimage before computing ARI and VOI？ @HuXiaoling @Zep4yrWang

[HuXiaoling]

Yes. I would recommend referring to this page for more detailed information: https://scikit-image.org/docs/stable/api/skimage.metrics.html