search

NVlabs / ODISE

Official PyTorch implementation of ODISE: Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models [CVPR 2023 Highlight]
https://arxiv.org/abs/2303.04803
Other
845 stars 45 forks source link

Abut the different of code and paper #27

Open Yi-Qi638 opened 1 year ago

Yi-Qi638 commented 1 year ago

The paper claims that ODISE freezes the Denoising Unet. However, upon inspecting the code from ODISE's "ldm.py" file, I encountered some aspects that left me uncertain about the actual freezing status of the Unet. This code is from ODISE/odise/modeling/meta_arch/ldm.py 974 20230626-103552

xvjiarui commented 1 year ago

Hi @Wangyiqi Since we need to back propagate the gradient through UNet ( we are not changing the UNet feature, we just need the grad to optimizer the MLP of implicit captioner), so we need to set the grad before we forward.