Can't train RTMDet on images with width or height greater than 1024

[GeorgePearse]

Describe the bug

Receive this error for any image size above 1024 (e.g. 1025) but none below.

Traceback (most recent call last):
  File "/data/ljh/anaconda3/envs/open-mmlab-new/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/data/ljh/anaconda3/envs/open-mmlab-new/lib/python3.10/site-packages/mmengine/optim/optimizer/optimizer_wrapper.py", line 351, in optim_context
    yield
  File "/data/ljh/anaconda3/envs/open-mmlab-new/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py", line 114, in train_step
    losses = self._run_forward(data, mode='loss')  # type: ignore
  File "/data/ljh/anaconda3/envs/open-mmlab-new/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py", line 326, in _run_forward
    results = self(**data, mode=mode)
  File "/data/ljh/anaconda3/envs/open-mmlab-new/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/ljh/mmdetection3/mmdet/models/detectors/base.py", line 92, in forward
    return self.loss(inputs, data_samples)
  File "/data/ljh/mmdetection3/mmdet/models/detectors/single_stage.py", line 77, in loss
    x = self.extract_feat(batch_inputs)
  File "/data/ljh/mmdetection3/mmdet/models/detectors/single_stage.py", line 148, in extract_feat
    x = self.neck(x)
  File "/data/ljh/anaconda3/envs/open-mmlab-new/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/ljh/mmdetection3/mmdet/models/necks/cspnext_pafpn.py", line 153, in forward
    torch.cat([upsample_feat, feat_low], 1))
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 104 but got size 103 for tensor number 1 in the list.

Looks like all of the model range are designed for 640x640 so I'm not sure what I could change in the neck to allow for larger images.

[ApoorvaSuresh-Audi]

Hi, did you find a workaround for this issue?

[GeorgePearse]

Nope, spent a while trying to tweak different parts of the architecture but just hit other problems.

Will continue to try whenever I have a bit of spare time.

[GeorgePearse]

Bump

[yanhaoerer]

Excuse me. Have you solved the problem yet?

[GeorgePearse]

No I've not, tried a variety of sizes for different parts of the model. Couldn't work out the bottleneck, very curious to find out the resolution.

[pd162]

Try to set keep_ratio as False in train_pipeline in config file.

[GeorgePearse]

Hi @pd162 that's not much of a fix for me, because I definitely do want to keep the same ratio (and all of the transforms I'm trying would do so anyway).

[Edwin-Koh1]

@GeorgePearse did you find the solution? Encountering the same issue now

[GeorgePearse]

Can't really remember.

But for anything you try, make sure the dimensions are divisable by 32.

They have a padding operation specifically for that.