search

Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.5k stars 1.01k forks source link

RandSpatialCropSamplesd #7828

Open wxr521314 opened 3 weeks ago

wxr521314 commented 3 weeks ago

Hi，I want to use batch_size = 2 for training. When using CenterSpatialCropd(keys=["image", "label"],roi_size=(224, 224, 224)), I can train the network, but when I use RandSpatialCropSamplesd(keys=["image", "label"],roi_size=(224, 224, 224), random_size=False, num_samples=2),ResizeWithPadOrCropd( keys=["image", "label"], spatial_size=224, mode='constant', ), the following error occurred.

collate/stack a list of tensors E: stack expects each tensor to be equal size, but got [1, 275, 386, 386] at entry 0 and [1, 275, 380, 380] at entry 2, shape [torch.Size([1, 275, 386, 386]), torch.Size([1, 275, 386, 386]), torch.Size([1, 275, 380, 380]), torch.Size([1, 275, 380, 380])] in collate([tensor([[[[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], ..., [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     ...,

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]]]]), tensor([[[[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     ...,

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]]]]), tensor([[[[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     ...,

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]]]]), tensor([[[[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     ...,

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]],

     [[0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      ...,
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.],
      [0., 0., 0.,  ..., 0., 0., 0.]]]])])

collate dict key "image_meta_dict" out of 4 keys

collate dict key "sizeof_hdr" out of 43 keys

collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "extents" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "session_error" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "dim_info" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "dim" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "intent_p1" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "intent_p2" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "intent_p3" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "intent_code" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "datatype" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "bitpix" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "slice_start" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "pixdim" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "vox_offset" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "scl_slope" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "scl_inter" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "slice_end" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "slice_code" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "xyzt_units" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "cal_max" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "cal_min" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "slice_duration" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "toffset" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "glmax" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "glmin" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "qform_code" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "sform_code" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "quatern_b" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "quatern_c" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "quatern_d" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "qoffset_x" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "qoffset_y" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "qoffset_z" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "srow_x" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "srow_y" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "srow_z" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "affine" out of 43 keys collate/stack a list of tensors collate dict key "original_affine" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "as_closest_canonical" out of 43 keys collate dict key "spatial_shape" out of 43 keys collate/stack a list of numpy arrays collate/stack a list of tensors collate dict key "space" out of 43 keys collate dict key "original_channel_dim" out of 43 keys collate dict key "filename_or_obj" out of 43 keys Traceback (most recent call last): File "/mnt/store/zhengsch/Ostia_landmark/tools/train.py", line 572, in Trainer().run() File "/mnt/store/zhengsch/Ostia_landmark/tools/train.py", line 332, in run self.train_step(epoch) File "/mnt/store/zhengsch/Ostia_landmark/tools/train.py", line 392, in train_step for i, batch_data in enumerate(self.trn_dl): File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data() File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data) File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise() File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise raise exception RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/data/utils.py", line 516, in list_data_collate ret = collate_fn(data) File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/data/utils.py", line 478, in collate_meta_tensor return {k: collate_meta_tensor([d[k] for d in batch]) for k in elem_0} File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/data/utils.py", line 478, in return {k: collate_meta_tensor([d[k] for d in batch]) for k in elem_0} File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/data/utils.py", line 483, in collate_meta_tensor return default_collate(batch) File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 140, in defaultcollate out = elem.new(storage).resize(len(batch), *list(elem.size())) RuntimeError: Trying to resize storage that is not resizable

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index) File "/opt/miniforge3/envs/monai/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch return self.collate_fn(data) File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/data/utils.py", line 696, in pad_list_data_collate return PadListDataCollate(method=method, mode=mode, **kwargs)(batch) File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/transforms/croppad/batch.py", line 114, in call return list_data_collate(batch) File "/home/zhengsch/.local/lib/python3.9/site-packages/monai/data/utils.py", line 529, in list_data_collate raise RuntimeError(re_str) from re RuntimeError: Trying to resize storage that is not resizable

wxr521314 commented 3 weeks ago

How can I solve this?

KumoLiu commented 2 weeks ago

Hi @wxr521314, from the error message got [1, 275, 386, 386] at entry 0 and [1, 275, 380, 380] at entry 2, it shows that the input shape are not the same after transforms, please check the shape of the data after the transforms. Thanks.