xmu-xiaoma666 / External-Attention-pytorch

🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐
MIT License
11.4k stars 1.93k forks source link

MobileViT-einops.EinopsError: Error while processing rearrange-reduction pattern "b d (h ph) (w pw) -> b (ph pw) (h w) d". #44

Open JiangMei0201 opened 2 years ago

JiangMei0201 commented 2 years ago

Thanks for your great repo. I changed some models'backbone into MobileViT, but it were wrong in traning. I checked the feature size carefully and I don‘t know how to solve it. Looking for your reply and thanks a lot.

Jyy01 commented 2 years ago

In External-Attention-pytorch/model/backbone/MobileViT.py,line134,you can change it. nn.Conv2d(inp, hidden_dim, kernel_size=1, stride=self.stride, bias=False),

JiangMei0201 commented 2 years ago

In External-Attention-pytorch/model/backbone/MobileViT.py,line134,you can change it. nn.Conv2d(inp, hidden_dim, kernel_size=1, stride=self.stride, bias=False), 谢谢您的帮助!!!我按照您的建议修改line 134,这个错误解决了。但是出现了新的错误如下: raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size)) ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256, 1, 1]) 我尝试了drop_last=True和改大batchsize,仍未解决。请问您遇到过这个错误吗?

Jyy01 commented 2 years ago

看起来像是你的输入数据的问题,你可以先定义一个随机数,然后调试一下,观察每一层的输出结果是否正确。 input = torch.randn(1, 3, 256, 256)

WangQvQ commented 1 year ago

@JiangMei0201 您好我也遇到了同样的问题,请问您解决了嘛