[Bug] [SOLO v2] (-215:Assertion failed) !dsize.empty() in function 'cv2.resize'

[hav4ik]

When applying SOLOv2 to samples with large number of instance annotations (>512), the referenced function below fails with OpenCV error message: (-215:Assertion failed) !dsize.empty() in function 'cv2.resize'. https://github.com/aim-uofa/AdelaiDet/blob/a647498e8f6f6656e59c7e439fd58d6d72c278b3/adet/modeling/solov2/utils.py#L27

According to issue https://github.com/opencv/opencv/issues/14770 in OpenCV github repository, this problem is caused by the hard limit in OpenCV containers, which is 512 channels. Currently, I've fixed this issue with a hack in my fork: https://github.com/hav4ik/AdelaiDet/commit/6ed9c1e1a25a3e25dddfa858ce0f219a30593ce2. Alternative solutions would be to use skimage transform function: https://scikit-image.org/docs/dev/api/skimage.transform.html

There are some other bugs in SOLOv2, I'll file other issues for them and make a PR with my fixes (or hacks) soon.

[WeiYangBin]

I used the method you said, but the report was wrong

Traceback (most recent call last):
  File "tools/train_net.py", line 235, in <module>
    args=(args,),
  File "/home/xair/anaconda3/envs/solov2/lib/python3.7/site-packages/detectron2/engine/launch.py", line 82, in launch
    main_func(*args)
  File "tools/train_net.py", line 223, in main
    return trainer.train()
  File "tools/train_net.py", line 102, in train
    self.train_loop(self.start_iter, self.max_iter)
  File "tools/train_net.py", line 91, in train_loop
    self.run_step()
  File "/home/xair/anaconda3/envs/solov2/lib/python3.7/site-packages/detectron2/engine/defaults.py", line 494, in run_step
    self._trainer.run_step()
  File "/home/xair/anaconda3/envs/solov2/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 273, in run_step
    loss_dict = self.model(data)
  File "/home/xair/anaconda3/envs/solov2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/xair/wyb/AdelaiDet/adet/modeling/solov2/solov2.py", line 136, in forward
    targets = self.get_ground_truth(gt_instances, mask_feat_size)
  File "/home/xair/anaconda3/envs/solov2/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/xair/wyb/AdelaiDet/adet/modeling/solov2/solov2.py", line 164, in get_ground_truth
    mask_feat_size=mask_feat_size)
  File "/home/xair/wyb/AdelaiDet/adet/modeling/solov2/solov2.py", line 217, in get_ground_truth_single
    gt_masks = imrescale(gt_masks, scale=1./output_stride)
  File "/home/xair/wyb/AdelaiDet/adet/modeling/solov2/utils.py", line 136, in imrescale
    rescaled_img = imresize(img, new_size, interpolation=interpolation)
  File "/home/xair/wyb/AdelaiDet/adet/modeling/solov2/utils.py", line 60, in imresize
    interpolation=interp_codes[interpolation])
cv2.error: OpenCV(4.5.5) :-1: error: (-5:Bad argument) in function 'resize'
> Overload resolution failed:
>  - Layout of the output array dst is incompatible with cv::Mat
>  - Expected Ptr<cv::UMat> for argument 'dst'

[yyhuang01]

I met the similar problem: [04/29 21:38:45 d2.utils.events]: eta: 2 days, 11:12:44 iter: 19 total_loss: 3.773 loss_ins: 2.984 loss_cate: 0.7884 time: 0.9492 last_time: 0.5499 data_time: 0.0695 last_data_time: 0.0796 lr: 0.0002881 max_mem: 5816M Traceback (most recent call last): File "/root/autodl-tmp/AdelaiDet/tools/train_net.py", line 310, in launch( File "/root/autodl-tmp/detectron2/detectron2/engine/launch.py", line 84, in launch main_func(args) File "/root/autodl-tmp/AdelaiDet/tools/train_net.py", line 304, in main return trainer.train() File "/root/autodl-tmp/AdelaiDet/tools/train_net.py", line 171, in train self.train_loop(self.start_iter, self.max_iter) File "/root/autodl-tmp/AdelaiDet/tools/train_net.py", line 160, in train_loop self.run_step() File "/root/autodl-tmp/detectron2/detectron2/engine/defaults.py", line 496, in run_step self._trainer.run_step() File "/root/autodl-tmp/detectron2/detectron2/engine/train_loop.py", line 310, in run_step loss_dict = self.model(data) File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/root/autodl-tmp/AdelaiDet/adet/modeling/solov2/solov2.py", line 136, in forward targets = self.get_ground_truth(gt_instances, mask_feat_size) File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/root/autodl-tmp/AdelaiDet/adet/modeling/solov2/solov2.py", line 163, in get_ground_truth self.get_ground_truth_single(img_idx, gt_instances, File "/root/autodl-tmp/AdelaiDet/adet/modeling/solov2/solov2.py", line 217, in get_ground_truth_single gt_masks = imrescale(gt_masks, scale=1./output_stride) File "/root/autodl-tmp/AdelaiDet/adet/modeling/solov2/utils.py", line 121, in imrescale rescaled_img = imresize(img, new_size, interpolation=interpolation) File "/root/autodl-tmp/AdelaiDet/adet/modeling/solov2/utils.py", line 45, in imresize resized_img = cv2.resize( cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgproc/src/resize.cpp:3789: error: (-215:Assertion failed) !dsize.empty() in function 'resize' I used my own datasets which has only one channel, but the same datasets is useful in CondInst. it also confused me that the model had begun training, then the problem occurred. i have no idea how to fix it .