Closed nkcdy closed 5 years ago
Yes, the batch size is 2.
nkcdy - think why there is x.squeeze!1) in input and x.unsqueeze!1) in output Also BatchNorm in small batch size seems to give lower noise.
I can't figure out why there is a squeeze at the input and an unaqueeze at the output. I also can't see any difference in the data before and after squeeze except the transpose operation during training. Could you please provide some prompt?
squeeze and unsqueeze only add or remove dimensions, please refer to the doc of pytorch for more details
yes, I know exactly the meaning of squeeze and unsqueeze. But Barby1138 implied there seems to exist some relationhsip between these two operations and the small size batchmorm. I just got confused.
They are not related
The paper said the batch size is 2 for 100k steps. Would it be too small for batch normalization? are you sure the batch size is 2 in your training?