TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'NoneType'>

[luhc228]

when I run

python get_roi.py -b 32 -s 512 512 --org_data "../../../kits19/data" --data "data" -r "runs/ResUNet/checkpoint/best.pth" -o "data/roi.json"

will raise this error

------subset----------- <torch.utils.data.dataset.Subset object at 0x7f7c289d7048>
------sampler----------- <torch.utils.data.sampler.SequentialSampler object at 0x7f7c269081d0>
------data_loader----------- <torch.utils.data.dataloader.DataLoader object at 0x7f7c26908080>
eval/test:   0%|                                                       | 0/90 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "get_roi.py", line 185, in <module>
    get_roi()
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "get_roi.py", line 181, in get_roi
    get_roi_from_resunet(batch_size, num_gpu, img_size, data_path, resume, roi_file, vis_intvl, num_workers)
  File "get_roi.py", line 120, in get_roi_from_resunet
    for batch_idx, data in enumerate(data_loader):
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 346, in __next__
    data = self.dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 75, in default_collate
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 75, in <dictcomp>
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/root/anaconda3/envs/kits19-challenge/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 82, in default_collate
    raise TypeError(default_collate_err_msg_format.format(elem_type))
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'NoneType'>

the related code is

  # get_roi.py   get_roi_from_resunet method
   ......
    subset = dataset.test_dataset
    print('------subset-----------', subset)
    case_slice_indices = dataset.test_case_slice_indices

    sampler = SequentialSampler(subset)
    print('------sampler-----------', sampler)
    data_loader = DataLoader(subset, batch_size=batch_size, sampler=sampler,
                             num_workers=num_workers, pin_memory=True)
    print('------data_loader-----------', data_loader)
    case = 0
    vol_output = []

    with tqdm(total=len(case_slice_indices) - 1, ascii=True, desc=f'eval/test', dynamic_ncols=True) as pbar:
        for batch_idx, data in enumerate(data_loader):
            print('-----------batch_idx------------', batch_idx)
            print('---------data-------------', data)
            imgs, idx = data['image'].cuda(), data['index']

Could you give me some help?

[luhc228]

[nitsaick]

It means DataLoader get None from Dataset. if __getitem() in torch.utils.data.Dataset return None, then you will get this error. You need to check __getitem() in KiTS19 class (dataset/kits19.py). Try to find why data is None. Maybe data path is wrong.

[amitjoshi24]

i nee to return None in some cases, how can i get this to workj