CAT-Seg's training setting

[BuRr-Lee]

i want to know how do you train the CAT-Seg model after replacing its clip with eva (trained using clipself method). especially, do you freeze the swin transformer backbone? do you finetune only attention layers in clip?

[wusize]

Hi, we use the model from OpenAI for the results on Cat-Seg. The training of Cat-Seg remain unchanged, where attention layers of CLIP are finetuned.

[BuRr-Lee]

you mean you first train the original OpenAI-pretrained CLIP (instead of EVA-CLIP) model with CLIPSelf method, then apply it to CAT-Seg?

[wusize]

you mean you first train the original OpenAI-pretrained CLIP (instead of EVA-CLIP) model with CLIPSelf method, then apply it to CAT-Seg?

Yeah. We need to make sure the comparison with Cat-Seg is under the fair setting.