why we need to requires_grad=True for batch input, which (not sure) may lead to out of memory error.

QinbinLi / MOON

Model-Contrastive Federated Learning (CVPR 2021)

MIT License

263 stars 56 forks source link

why we need to requires_grad=True for batch input, which (not sure) may lead to out of memory error. #5

Closed SunTongtongtong closed 3 years ago

SunTongtongtong commented 3 years ago

Hello,

Thanks for sharing the interesting work, I just have one question about the codehttps://github.com/QinbinLi/MOON/blob/main/main.py#L212

You set

x.requires_grad = True in main.py Line 212. I just cannot understand why it is needed. As I met memory leak when I implement this into my own framework. After I remove this line, it works.

Looking forwards to your reply.

QinbinLi commented 3 years ago

Hi @SunTongtongtong ,

Thanks for pointing it out. Indeed, since we only optimize the network's parameters here, there is no need to set x.requires_grad = True, which may require additional GPU memory. I have fixed it.