search

joe-siyuan-qiao / DetectoRS

DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution
Apache License 2.0
1.13k stars 176 forks source link

AssertionError: Caught AssertionError in DataLoader worker process 0. #48

Open lgldl opened 4 years ago

lgldl commented 4 years ago

We prepared the data followed by https://github.com/joe-siyuan-qiao/DetectoRS/tree/master/configs/htc, but when we train our own data with seg task, the error occured.

loading annotations into memory... Done (t=0.62s) creating index... index created! loading annotations into memory... Done (t=0.05s) creating index... index created! 2020-07-05 19:34:20,227 - mmdet - INFO - Start running, host: amax@amax, work_dir: /home/lgl/PycharmProjects/DetectoRS/work_dirs/building-seg 2020-07-05 19:34:20,227 - mmdet - INFO - workflow: [('train', 1)], max: 45 epochs Traceback (most recent call last): File "/home/lgl/PycharmProjects/DetectoRS/train_building.py", line 144, in main() File "/home/lgl/PycharmProjects/DetectoRS/train_building.py", line 140, in main meta=meta) File "/home/lgl/PycharmProjects/DetectoRS/mmdet/apis/train.py", line 111, in train_detector meta=meta) File "/home/lgl/PycharmProjects/DetectoRS/mmdet/apis/train.py", line 242, in _non_dist_train runner.run(data_loaders, cfg.workflow, cfg.total_epochs) File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/mmcv/runner/epoch_based_runner.py", line 122, in run epoch_runner(data_loaders[i], **kwargs) File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/mmcv/runner/epoch_based_runner.py", line 27, in train for i, data_batch in enumerate(data_loader): File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 345, in next data = self._next_data() File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data return self._process_data(data) File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data data.reraise() File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/torch/_utils.py", line 394, in reraise raise self.exc_type(msg) AssertionError: Caught AssertionError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch return self.collate_fn(data) File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/mmcv/parallel/collate.py", line 82, in collate for key in batch[0] File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/mmcv/parallel/collate.py", line 82, in for key in batch[0] File "/home/lgl/anaconda3/envs/DetectoRS/lib/python3.6/site-packages/mmcv/parallel/collate.py", line 47, in collate assert batch[i].size(dim) == sample.size(dim) AssertionError

joe-siyuan-qiao commented 4 years ago

I would recommend first trying to run experiments on COCO. If things go well, then move towards custom datasets. It's hard to diagnose when custom datasets are used. If you believe there are some bugs, please try to pull the latest mmdetection / mmcv to see if they have fixed it.