loli / medpy

Medical image processing in Python
http://loli.github.io/medpy/
GNU General Public License v3.0
559 stars 136 forks source link

ASSD equation is wrong #108

Closed Jingnan-Jia closed 1 year ago

Jingnan-Jia commented 1 year ago

Average symmetric surface distance (assd) seems implemented wrongly here.

In this line%20)), assd = (asd(A, B) + asd(B, A))/2, where A and B are two sets of points respresenting the reference and result. However, it is wrong.

The correct equaton of assd should be: assd = (asd(A, B) + asd(B, A))/(N_A + N_B), where N_A and N_B are the total pixel numbers of A and B.

A great number of academic papers used the second equation, for instance:

  1. Yeghiazaryan, Varduhi, and Irina D. Voiculescu. "Family of boundary overlap metrics for the evaluation of medical image segmentation." Journal of Medical Imaging 5.1 (2018): 015006. “The average symmetric surface distance (ASSD) is the average of all the distances from points on the boundary of machine segmented region to the boundary of the ground truth, and vice versa.”
  2. Heimann T, Ginneken B, Styner MA, et al. Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets. IEEE Transactions on Medical Imaging. 2009;28(8):1251–1265. "Average Symmetric Surface Distance: The average symmetric surface distance is given in millimeters and based on the surface voxels of two segmentations and . Surface voxels are defined by having at least one non-object voxel within their 18-neighborhood. For each surface voxel of , the Euclidean distance to the closest surface voxel of is calculated using the approximate nearest neighbor technique [26] and stored. In order to provide symmetry, the same Process. is applied from the surface voxels of to . The average symmetric surface distance is then defined as the average of all stored distances, which is 0 for a perfect segmentation."
  3. Ruskó, László, György Bekes, and Márta Fidrich. "Automatic segmentation of the liver from multi-and single-phase contrast-enhanced CT images." Medical Image Analysis 13.6 (2009): 871-882. "For each contour voxel of the segmented volume the Euclidean distance from the closest contour voxel of the reference volume is computed. This computation is also performed for the contour voxels of the reference volume. The metric represents the average of these distances."
loli commented 1 year ago

Well noticed! I made here the classical mistake of taking the mean of some means - which works well, when both are of the same size, but not in the present case, where the surface voxel of A and B might differ in number.

Luckily, the numbers won't we off by much as long as |A| and |B| are of similar size.

Will close after merge. Thank you for contributing!