Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.5k stars 1.01k forks source link

Bug in the end index computation of RandCropBoxByPosNegLabeld.generate_fg_center_boxes_np #7824

Open AndersMollgaard opened 3 weeks ago

AndersMollgaard commented 3 weeks ago

Describe the bug The RandCropBoxByPosNegLabeld.generate_fg_center_boxes_np method computes a wrong end index for the box when self.whole_box is True.

The code is as follows:

# extended box start
extended_boxes[:, axis] = boxes_stop[:, axis] - self.spatial_size[axis] // 2 - 1
extended_boxes[:, axis] = np.minimum(extended_boxes[:, axis], boxes_start[:, axis])
# extended box stop
extended_boxes[:, axis + spatial_dims] = extended_boxes[:, axis] + self.spatial_size[axis] // 2
extended_boxes[:, axis + spatial_dims] = np.maximum(
    extended_boxes[:, axis + spatial_dims], boxes_stop[:, axis]
)

When the box is smaller than half the cropping size, this code will just create an extended box which shares the end index of the input box. Instead, the first line of the extended box stop computation should be

# extended box stop
extended_boxes[:, axis + spatial_dims] = boxes_start[:, axis] + self.spatial_size[axis] // 2

This will ensure that the end index of the foreground sampling region is indeed extended, but no further than still fully containing the original box.

To Reproduce Run the following code:

>>> crop_size = (20, 20)
>>> transform = RandCropBoxByPosNegLabeld(image_keys=['images'], box_keys='boxes', label_keys='labels', whole_box=True, spatial_size=crop_size)
>>> transform.spatial_size = crop_size
>>> boxes = np.array([[25, 25, 30, 30]])
>>> transform.generate_fg_center_boxes_np(boxes, image_size=(50, 50))
array([[19, 19, 30, 30]])

Expected behavior I expect the generated foreground boxes (the extended boxes) to also be extended in the direction of the end index. The box should be extended as far as possible, while still including the original box when placing the center of the cropping region in any of the pixels of the extended box. In this case, it should be something like array([[19, 19, 35, 35]]).

I say "something like" since array([[21, 21, 35, 35]]) would probably be even better to ensure that the whole box is included. If I understand the downstream cropping code correctly, it will prefer to "extend" left over right, when the region is even sized. In the above case it will therefore include 10 pixels towards the start index and only 9 pixels towards the end index (relative to the center).

Environment

================================ Printing MONAI config...

MONAI version: 0+untagged.3047.g4029c42.dirty Numpy version: 1.26.4 Pytorch version: 2.3.0+cu121 MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False MONAI rev id: 4029c422f1da84cdf5e4ce546dbff33245cbe025 MONAI file: /home//repos/MONAI/monai/init.py

Optional dependencies: Pytorch Ignite version: 0.4.11 ITK version: 5.4.0 Nibabel version: 5.2.1 scikit-image version: 0.22.0 scipy version: 1.13.1 Pillow version: 10.3.0 Tensorboard version: 2.16.2 gdown version: 5.2.0 TorchVision version: 0.18.0+cu121 tqdm version: 4.66.4 lmdb version: 1.4.1 psutil version: 5.9.0 pandas version: 2.2.2 einops version: 0.8.0 transformers version: 4.40.2 mlflow version: 2.13.1 pynrrd version: 1.0.0 clearml version: 1.16.2rc0

For details about installing the optional dependencies, please visit: https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================ Printing system config...

System: Linux Linux version: Ubuntu 20.04.6 LTS Platform: Linux-5.15.0-107-generic-x86_64-with-glibc2.31 Processor: x86_64 Machine: x86_64 Python version: 3.10.14 Process name: pt_main_thread Command: ['python', '-c', 'import monai; monai.config.print_debug_info()'] Open files: [] Num physical CPUs: 8 Num logical CPUs: 16 Num usable CPUs: 16 CPU usage (%): [17.3, 3.2, 2.0, 2.4, 2.3, 2.2, 3.5, 2.6, 3.4, 2.9, 3.2, 2.0, 2.7, 3.0, 3.1, 5.3] CPU freq. (MHz): 2 Load avg. in last 1, 5, 15 mins (%): [10.8, 4.4, 3.2] Disk usage (%): 89.8 Avg. sensor temp. (Celsius): UNKNOWN for given OS Total physical memory (GB): 31.1 Available memory (GB): 19.5 Used memory (GB): 9.5

================================ Printing GPU config...

Can't initialize NVML Num GPUs: 0 Has CUDA: False cuDNN enabled: True NVIDIA_TF32_OVERRIDE: None TORCH_ALLOW_TF32_CUBLAS_OVERRIDE: None cuDNN version: 8902

Additional context

I will provide a pull request today.