search

JiazuoYu / MoE-Adapters4CL

Code for paper "Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters" CVPR2024
119 stars 6 forks source link

Label Smooth limited the data set can be used in this method，how about performance without the Label Smooth #5

Closed AHideoKuzeA closed 1 month ago

AHideoKuzeA commented 4 months ago

Thanks to you and your colleagues for the excellent algorithm as well as the complete code, I noticed that you used the label smooth trick in the methods and code. However, for many image text-to-data (e.g. COCO Caption, WIT, etc.), the text is mostly given in a descriptive way, and the extra effort of extracting the kinds of labels is significant, resulting in these datasets not being usable for the proposed method. Are there ablation experiments to demonstrate the improvement of label smooth for the overall performance, and whether the performance of the method can also be guaranteed for descriptive text without labels.

JiazuoYu commented 4 months ago

Thank you for your interest in our work.

  1. The label smooth of this work follows the same settings of most previous methods, without special adjustments. And You can explore the verification process you're interested in based on the Repo.
  2. Currently, our classification tasks primarily involve simple class name text and have not yet extended fine-tuning CLIP to more descriptive sentences. We are currently conducting follow-up research on more open captions, and we invite you to stay tuned for our upcoming work.