File "D:\code\TransNeXt\transnext_native.py", line 234, in forward attn_pool = q_norm_scaled @ c.transpose(-2, -1) + pool_bias RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [147, 24] but got: [147, 49].

[aoe1223]

Hello, I encountered the following error while trying to train a classification model on the CIFAR-10 dataset using this project: q_norm_scaled.shape=[49, 3, 16, 24], c=F.normalize(k_pool, dim=-1), c.shape=[49, 3, 24, 49]. The dimensions of these two matrices satisfy the conditions for matrix multiplication, but I still received an error. File "D:\code\TransNeXt\transnext_native.py", line 234, in forward attn_pool = q_norm_scaled @ c.transpose(-2, -1) + pool_bias RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [147, 24] but got: [147, 49].

[DaiShiResearch]

It appears that the inconsistency between the size of the input image during training and the model’s preset value is causing the relative position bias to be unable to be added. It is recommended to print out the size of the input image and the size of the relative position bias for each layer to determine whether the operation of the model meets your expectations.