search

chaiyujin / glow-pytorch

pytorch implementation of openai paper "Glow: Generative Flow with Invertible 1×1 Convolutions"
MIT License
507 stars 80 forks source link

Inplace operation error occurs with batch size 1 #2

Closed wmjung closed 5 years ago

wmjung commented 6 years ago

Hi yujin, I'm applying generative flow on 3D model. Your pytorch implementation is helpful. Due to the memory size, I ran the code with batch size 1, then inplace operation error occurs like below. (it is okay with batch size 2 or 4)

"RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation"

I tried to figure out where this error comes from, but failed. Do you have any idea?

Thanks in advance Wonmo,

chaiyujin commented 6 years ago

Hi Wonmo, Have you ever tried to change nn.ReLU(inplace=True) into nn.ReLU(inplace=False)? In thops.py, squeeze_() and scatter_() could be problem too. It seems that there are no other inplace operations in my code.

I cannot run codes these days. I will fix this later. You can try to modify the above three mentioned places first. Thanks.

wmjung commented 6 years ago

Thank you for your reply. I'll try with those modifications :)

chaiyujin commented 6 years ago

I tried to modify some places and run on my cpu, but failed to figure out the problem. I have no idea what happened. It could be a small bug. If you found something, please tell me. Thank you.

kolchinski commented 5 years ago

I was getting a similar error when using some of the code - looks like it was coming from the in-place ops in the coupling layers (e.g. z2 += z1). Just sent a PR to fix it.

chaiyujin commented 5 years ago

@kolchinski Thanks a lot. @wmjung It seems the bugs are fixed.