fundamentalvision / Deformable-DETR

Deformable DETR: Deformable Transformers for End-to-End Object Detection.
Apache License 2.0
3.14k stars 513 forks source link

batch % im2col_step_ == 0 INTERNAL ASSERT FAILED #224

Open bio-mlhui opened 7 months ago

bio-mlhui commented 7 months ago

A video of 154 frames (batch_size=1, t=154) is flattend(bt=154, c=64, hw_sum=(11^2+22^2+44^2)) and input to Deformable Attention, but I encounted batch % im2colstep == 0 INTERNAL ASSERT FAILED at "/home/xuhuihui/workspace/Deformable-DETR/models/ops/src/cuda/ms_deform_attn_cuda.cu":52, please report a bug to PyTorch. batch(%d) must divide im2col_step(%d)15464.

In ms_deform_attn_cuda.cu, one line is const int im2colstep = std::min(batch, im2col_step);. Should I increase im2col_step to bigger than 154, like 256?

when i set imcol_step to 256, and no error encounted.