Extension of RandCropByPosNegLabeld to extract patches with positive voxels anywhere in the patch

[martaranzini]

Is your feature request related to a problem? Please describe. I would like to make sure I compose my batch with both positive and negative examples. However, I do have a patch size corresponding to the whole 2D slice from a 3D volume, and the labelled object might not necessarily be at the centre of it. I understand that the RandCropByPosNegLabeld transform goes towards the direction of balancing samples, but selects the patch as positive/negative based on the patch centre voxel only.

Describe the solution you'd like Would it be possible to extend the RandCropByPosNegLabeld transform so as to consider a patch positive based on the presence of foreground label anywhere in the patch?

Additional context I have noticed there is only a dict implementation of this transform and not an array one. Is there a particular reason for it?

[Nic-Ma]

Hi @martaranzini ,

Thanks for your feedback!

1. The logic is to randomly select a foreground point from image, then use it as center crop. Maybe you can pad your image first?
2. Because this transform needs to get label from label data and crop on both label and image. I feel it's not convenient for array level to work on several data together. Of course, if you are interested, welcome to submit a PR for array level transform.

Thanks.

[martaranzini]

Hi @Nic-Ma,

Thank you for getting back to me and apologies for my belated reply.

Thank you for clarifying the logic of the balanced random cropping. In my case I do not necessarily want to have the foreground object in the centre of the patch, so I am afraid the padding solution would not be suitable. I was thus wondering whether there is any plan to extend this transform to a more general case with the foreground object being anywhere in the patch?

Many thanks.

[Nic-Ma]

Hi @martaranzini ,

I think you mean to do 3 steps: (1) randomly select a point(based on pos/neg ratio), (2) select a random distance(0, crop_size/2), (3) use point + distance as the patch center to crop patch. Is it right?

BTW, I will add Array level PosNegCrop soon in https://github.com/Project-MONAI/MONAI/issues/583.

Thanks.

[martaranzini]

Hi @Nic-Ma,

Sorry, I think I haven't stated my problem very clearly in relation to the transform. The transform checks only the central voxel to be either foreground or background, is this right? If the central voxel is background, there is still some chance that the selected patch still contains foreground voxels somewhere. In my case I would like to check whether the whole patch contains any foreground labels or not. So probably I would reformulate the steps as the following: (1) Extract a random patch (2) Determine whether patch has any foreground voxels or is just full background (e.g. by counting the number of voxels with ground truth segmentation == foreground label) (3) Decide whether to keep it in the batch based on the pos/neg ratio. So if you want, I would consider as "positive" a patch that has some foreground voxels anywhere, and "negative" a patch that has none (rather than looking at the central voxel only).

Just to give you a bit of context, I am working on brain segmentation in fetal MRI and because the field of view typically includes also the body of the fetus and the womb, there is quite some imbalance in foreground (brain) and background (all the rest). So I would like to compose my batch in a balanced way. However, I still want to keep some patches with just background as there are some confounding structures I want the network to see and learn they are not part of the brain. Hope this helps?

Thanks.

Ps thank you for implementing the array version as well.

[Nic-Ma]

Hi @martaranzini ,

Thanks for your explanation. To solve your unbalanced dataset issue, I think maybe you can try the image_key and image_threshold parameters in RandCropByPosNegLabeld transform? if image_key is not None, use label == 0 & image > image_threshold to select the negative sample(background) center. so the crop center will only exist on valid image area. And you can refer to the example in: https://github.com/Project-MONAI/MONAI/blob/master/examples/notebooks/spleen_segmentation_3d.ipynb

RandCropByPosNegLabeld(keys=['image', 'label'], label_key='label', size=(96, 96, 96),
    pos=1, neg=1, num_samples=4, image_key='image', image_threshold=0)

Thanks.

[Nic-Ma]

And BTW, I submitted ticket: https://github.com/Project-MONAI/MONAI/issues/586 to track your new features request of PosNegRatio crop.

Thanks.

[wyli]

closing this in favour of https://github.com/Project-MONAI/MONAI/issues/586

[Nic-Ma]

Hi @martaranzini ,

Mark: I added Array version PosNeg crop in PR: https://github.com/Project-MONAI/MONAI/pull/628. Thanks.