After change the batch size

I found in https://github.com/pytorch/pytorch/issues/47163 starhiking mentioned

I met the problem in distributed training with bacthsize > 1. In batchsize = 1 or single gpu, it won't occur. I thought the problem caused by transpose or permute. When I delete the transpose or permute, or add .contiguous() after above function, it goes work. Thus I suspect transpose caused the gradients stride wrong. Need to be contigous. Before: vit_ll = eval('self.vit'+str(i)).forward_wo_cls(x).transpose(1,2).view([x.shape[0],-1,x.shape[2:]]) after: (fixed) vit_ll = eval('self.vit'+str(i)).forward_wo_cls(x).transpose(1,2).contiguous().view([x.shape[0],-1,x.shape[2:]])

in https://blog.csdn.net/m0_67497308/article/details/123705898 he mentioned

原因是因为输入tensor进行过transpose或permute等变形操作后，内部的内存地址变得不连续导致的。处理方式很简单，在tensor进行transpose或permute等变形操作后，加一句语句.contiguous()，可使内存地址变连续。

But I don't know how to fix it. Could someone help me with this?

huaidanquede / MUSE-Speech-Enhancement

After change the batch size #2