Open Yi-Qi638 opened 1 year ago
Hi @Wangyiqi Since we need to back propagate the gradient through UNet ( we are not changing the UNet feature, we just need the grad to optimizer the MLP of implicit captioner), so we need to set the grad before we forward.
The paper claims that ODISE freezes the Denoising Unet. However, upon inspecting the code from ODISE's "ldm.py" file, I encountered some aspects that left me uncertain about the actual freezing status of the Unet. This code is from ODISE/odise/modeling/meta_arch/ldm.py 974![20230626-103552](https://github.com/NVlabs/ODISE/assets/12243771/c32da8af-956b-469a-a55b-1cfe7117e231)