search

wkcn / TinyCLIP

[ICCV2023] TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
https://github.com/microsoft/Cream/tree/main/TinyCLIP
Other
68 stars 4 forks source link

Tiny CLIP performance not upto mark #6

Closed adi070701 closed 1 month ago

adi070701 commented 1 month ago

Hi, I was testing out the performance for the image branch of TinyCLIP-ViT-40M-32-Text-19M-LAION400M and comparing it to vanilla clip-vit-base-patch32. Although TinyCLIP has half the parameters as clip-vit-base-paths32 it is only giving a 10% improvement to inference speed. Is this expected?

wkcn commented 1 month ago

Hi @adi070701 , thanks for your attention to our work!

The reason may be that the batch size is relatively small, resulting in underutilization of the hardware.

You can try using ONNX Runtime or PyTorch JIT for model inference while increasing the batch size.