search

WXinlong / SOLO

SOLO and SOLOv2 for instance segmentation, ECCV 2020 & NeurIPS 2020.
Other
1.7k stars 306 forks source link

RuntimeError: There were no tensor arguments to this function #218

Open DongGuangchang opened 2 years ago

DongGuangchang commented 2 years ago

Traceback (most recent call last): File "tools/train.py", line 125, in main() File "tools/train.py", line 121, in main timestamp=timestamp) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/apis/train.py", line 111, in train_detector timestamp=timestamp) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/apis/train.py", line 297, in _non_dist_train runner.run(data_loaders, cfg.workflow, cfg.total_epochs) File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/runner.py", line 364, in run epoch_runner(data_loaders[i], kwargs) File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/runner.py", line 268, in train self.model, data_batch, train_mode=True, kwargs) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/apis/train.py", line 78, in batch_processor losses = model(data) File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, *kwargs) File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(inputs[0], kwargs[0]) File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, *kwargs) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/core/fp16/decorators.py", line 49, in new_func return old_func(args, kwargs) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/models/detectors/base.py", line 142, in forward return self.forward_train(img, img_meta, kwargs) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/models/detectors/single_stage_ins.py", line 78, in forward_train *loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore) File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/models/anchor_heads/solov2_head.py", line 249, in loss loss_ins = torch.cat(loss_ins).mean() RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors. Available functions are [CUDATensorId, CPUTensorId, VariableTensorId]

I encountered a problem during training. How can I solve it? thanks！

biendltb commented 2 years ago

I hit the same issue. The problem is that there were empty predictions from the model for that batch which returns an empty list of losses. Not sure how this could be solved. This is a temporary solution that you could try (added to SOLO/mmdet/models/anchor_heads/solov2_head.py:249):

        if len(loss_ins) == 0:
            loss_ins = torch.tensor(0.0, device=gt_bbox_list[0].device)
        else:
            loss_ins = torch.cat(loss_ins).mean()
            loss_ins = loss_ins * self.ins_loss_weight