search

isl-org / lang-seg

Language-Driven Semantic Segmentation
MIT License
691 stars 84 forks source link

freeze CLIP TextEncoder #11

Closed Ahnsun closed 2 years ago

Ahnsun commented 2 years ago

Hi, the paper pointed out that we should freeze the text encoder of CLIP during training. I wonder how to achieve it and where is the corrresponding code. Thanks!

soskek commented 2 years ago

In my understanding (though I'm not an author), the optimizer selects which parameters to be updated. https://github.com/isl-org/lang-seg/blob/main/modules/lsegmentation_module.py#L119-L175 Because these lines do not select CLIP parameters (self.net.clip_pretrained), they are frozen.

Boyiliee commented 2 years ago

Thanks @soskek for answering this!

Best wishes for your research! @Ahnsun