Thanks to you and your colleagues for the excellent algorithm as well as the complete code, I noticed that you used the label smooth trick in the methods and code. However, for many image text-to-data (e.g. COCO Caption, WIT, etc.), the text is mostly given in a descriptive way, and the extra effort of extracting the kinds of labels is significant, resulting in these datasets not being usable for the proposed method. Are there ablation experiments to demonstrate the improvement of label smooth for the overall performance, and whether the performance of the method can also be guaranteed for descriptive text without labels.
The label smooth of this work follows the same settings of most previous methods, without special adjustments. And You can explore the verification process you're interested in based on the Repo.
Currently, our classification tasks primarily involve simple class name text and have not yet extended fine-tuning CLIP to more descriptive sentences. We are currently conducting follow-up research on more open captions, and we invite you to stay tuned for our upcoming work.
Thanks to you and your colleagues for the excellent algorithm as well as the complete code, I noticed that you used the label smooth trick in the methods and code. However, for many image text-to-data (e.g. COCO Caption, WIT, etc.), the text is mostly given in a descriptive way, and the extra effort of extracting the kinds of labels is significant, resulting in these datasets not being usable for the proposed method. Are there ablation experiments to demonstrate the improvement of label smooth for the overall performance, and whether the performance of the method can also be guaranteed for descriptive text without labels.