search

wkcn / TinyCLIP

[ICCV2023] TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
https://github.com/microsoft/Cream/tree/main/TinyCLIP
Other
65 stars 4 forks source link

Can I just distill the image? #1

Closed alexandsunny closed 7 months ago

alexandsunny commented 7 months ago

If I only distill the image side and not the text side, will the effect be better than distilling both sides?

wkcn commented 7 months ago

Hi @alexandsunny, thanks for your attention to our work!

In my experience, distilling both two branches is better than distilling a branch.

alexandsunny commented 7 months ago

Hi @alexandsunny, thanks for your attention to our work!

In my experience, distilling both two branches is better than distilling a branch.

Thank you for your patient answer！！！