Hausdorff and IoU are stopped changing while dice metric is decreasing

[al3ms]

Discussed in https://github.com/Project-MONAI/tutorials/discussions/1403

^{Originally posted by **al3ms** May 23, 2023} Hello everyone, and thanks for such a great framework. I need help using Hausdorff and IoU, where they stop learning when reaching a specific value! While the loss and dice metric keeps changing. surface_distance also has a problem since it is always equal to "inf". These are three measurements (it is the same for the following 30 epochs): ``` ********************Testing******************** Test_loss: 0.1801 Test_Metric: 0.6199 hausdorff: 24.8 surface_distance: inf IoU: 0.4916495382785797 current epoch: 1 current mean dice: 0.6199 best mean dice: 0.6199 at epoch: 1 ********************Testing******************** Test_loss: 0.1544 Test_Metric: 0.6456 hausdorff: 24.8 surface_distance: inf IoU: 0.4916495382785797 current epoch: 2 current mean dice: 0.6456 best mean dice: 0.6456 at epoch: 2 ********************Testing******************** Test_loss: 0.1111 Test_Metric: 0.6889 hausdorff: 24.8 surface_distance: inf IoU: 0.4916495382785797 current epoch: 3 current mean dice: 0.6889 best mean dice: 0.6889 at epoch: 3 ``` This is my code: ``` meanIoU = MeanIoU(reduction="mean") surfaceDistanceMetric = SurfaceDistanceMetric(include_background=True, symmetric=True, reduction="mean") hausdorffDistanceMetric = HausdorffDistanceMetric(include_background=True, percentile=95, reduction="mean") post_pred = Compose([AsDiscrete(argmax=True, to_onehot=2)]) post_label = Compose([AsDiscrete(to_onehot=2)]) ``` Inside testing part in each epoch: for t_data in test_loader: test_step += 1 # print("Now I will loop within Task ",i+1, " testing set:") test_volume = t_data["vol"] test_label = t_data["seg"] test_volume, test_label = (test_volume.to(device), test_label.to(device),) optimizer.zero_grad() test_outputs = model(test_volume) test_loss = loss_function(test_outputs, test_label) test_outputs2 = [post_pred(i) for i in decollate_batch(test_outputs)] test_label2 = [post_label(i) for i in decollate_batch(test_label)] surfaceDistanceMetric(test_outputs2, test_label2) hausdorffDistanceMetric(test_outputs2, test_label2) meanIoU(test_outputs2,test_label2) hd = hausdorffDistanceMetric.aggregate().item() sd = surfaceDistanceMetric.aggregate().item() iou = meanIoU.aggregate().item() hausdorffDistanceMetric.reset() surfaceDistanceMetric.reset() meanIoU.reset() print('hausdorff: ' , hd) print('surface_distance: ' , sd) print(f'IoU:',iou) Thanks

[al3ms]

This RuntimeWarning is related to the surface distance problem.

/usr/local/lib/python3.10/dist-packages/monai/metrics/surface_distance.py:163: UserWarning: the prediction of class 1 is all 0, this may result in nan/inf distance. warnings.warn(f"the prediction of class {c} is all 0, this may result in nan/inf distance.") /usr/local/lib/python3.10/dist-packages/monai/metrics/hausdorff_distance.py:170: UserWarning: the prediction of class 1 is all 0, this may result in nan/inf distance. warnings.warn(f"the prediction of class {c} is all 0, this may result in nan/inf distance.") /usr/local/lib/python3.10/dist-packages/numpy/lib/function_base.py:4486: RuntimeWarning: invalid value encountered in subtract diff_b_a = subtract(b, a)