Question about difference between stop gradient and *.requires_grad_(False)

threestudio-project / threestudio

A unified framework for 3D content generation.

Apache License 2.0

6.32k stars 480 forks source link

Question about difference between stop gradient and *.requires_grad_(False) #297

Closed Kitsunetic closed 1 year ago

Kitsunetic commented 1 year ago

I think simply freezing the pretrained diffusion model using *.requires_grad_(False) is simpler to implement than stop gradient operation, and I think both act same. So I'm not sure why the implementation in here are utilizing stop gradient operation instead of requires_grad_. Is there any other reason or is my understanding wrong? Can someone explain about this clearly?

thuliu-yt16 commented 1 year ago

It has nothing to do with threestudio.

But what did you mean by stop gradient? Is it .detach()? .detach() is used on intermediate variables to cut off the computation graph while .requires_grad_ is called on parameters to disable recording gradients.