mean dice:nan

[tiulpin]

Thank you for your code. I trained the model with the framework you provided: modified config.py to add paths to my preprocessed LUNA16 data (used your splits), then ran

python train.py

After 200 epochs the training was completed, I ran test.py, got those warnings:

/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2957: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
/lib/python3.6/site-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
mAP:  nan
/lib/python3.6/site-packages/numpy/core/_methods.py:135: RuntimeWarning: Degrees of freedom <= 0 for slice
  keepdims=keepdims)
/lib/python3.6/site-packages/numpy/core/_methods.py:105: RuntimeWarning: invalid value encountered in true_divide
  arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
/lib/python3.6/site-packages/numpy/core/_methods.py:127: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
mean dice:nan(nan)
mean dice (exclude fn):nan(nan)

And I got a NaN DICE value. Did you encounter such problems?

[tanghaotommy]

https://github.com/uci-cbcl/NoduleNet/blob/master/test.py#L96 Set the use_mask to True, otherwise the network will not predict segmentation mask. I forget to change this back

[tiulpin]

Changed this flag, now I get this error:

python -W ignore test.py eval --weight results/cross_val_test/rcnn/model/200.ckpt
[Loading model from results/cross_val_test/rcnn/model/200.ckpt]
out_dir nodule_net/results/cross_val_test
Total # of eval data 79
[0] Predicting 1.3.6.1.4.1.14519.5.2.1.6279.6001.603166427542096384265514998412 (281, 234, 306)
Traceback (most recent call last):
  File "test.py", line 130, in eval
    ap, dice = average_precision_score(gt_mask, pred_mask)
  File "python3.6/site-packages/sklearn/metrics/ranking.py", line 240, in average_precision_score
    average, sample_weight=sample_weight)
  File "python3.6/site-packages/sklearn/metrics/base.py", line 73, in _average_binary_score
    raise ValueError("{0} format is not supported".format(y_type))
ValueError: unknown format is not supported

And could you, please, provide the splits you used for your cross validation script? I see only 3/ dir in the repo. Thanks.

[tanghaotommy]

I will take a look at the error. The cross validation split is in https://github.com/uci-cbcl/NoduleNet/tree/master/split/cross_val

[tiulpin]

Thank you. I started debugging too.

[huliangxiao]

I got this when setting use_mask to Ture, I can't find 'average_precision' function.

[0] Predicting 1.3.6.1.4.1.14519.5.2.1.6279.6001.603166427542096384265514998412 (281, 234, 306) /home/hulx/PycharmProjects/Shown-NoduleNet-master/NoduleNet/net/layer/rcnn_nms.py:43: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument. probs = F.softmax(logits).cpu().data.numpy() /home/hulx/PycharmProjects/Shown-NoduleNet-master/NoduleNet/net/layer/rcnn_nms.py:117: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument. probs = F.softmax(logits).cpu().data.numpy() /home/hulx/anaconda3/envs/nas/lib/python3.6/site-packages/torch/nn/functional.py:2494: UserWarning: Default upsampling behavior when mode=trilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode)) /home/hulx/anaconda3/envs/nas/lib/python3.6/site-packages/torch/nn/functional.py:1351: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead. warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.") Traceback (most recent call last): File "/home/hulx/PycharmProjects/Shown-NoduleNet-master/NoduleNet/test1.py", line 128, in eval ap, dice = average_precision(gt_mask, pred_mask) NameError: name 'average_precision' is not defined

[tanghaotommy]

Oh, I see. Could you please comment this line and others that use ap and dice? average_precision was used as another metric to evaluate instance segmentation. However, we did not include that part of the code in the release. And related work usually only report mean dice so that is why we only include the mean dice calculation here.

[huliangxiao]

ok, I got it. Here are the codes.

            # compute average precisions
            ap, dice = average_precision(gt_mask, pred_mask)
            aps.append(ap)
            dices.extend(dice.tolist())

at line 128 in test.py

[liudongxu-hu]

could you send a LIDC - IDRI Radiologist Annotations/Segmentations (XML format) for me?1538378180@qq.com.thank you very much