search

FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs
MIT License
5.89k stars 426 forks source link

how to fine-tune Visualized BGE？ #829

Open CarllllWang opened 1 month ago

CarllllWang commented 1 month ago

Hi, I would like to know if it's possible to fine-tune the Visualized BGE? I am looking to perform a semantic similarity computation task, and I have some domain-specific data at my disposal. The data includes text for queries, texts and images for positive documents, as well as texts and images for negative documents. How should I go about fine-tuning the model for this purpose?

JUNJIE99 commented 1 month ago

Thank you for your attention!

Yes, you can use contrastive learning methods to fine-tune the visualized BGE on your dataset. We have conducted relevant downstream experiments, and the Visualized BGE has good generalization ability.

As I am quite busy recently, I might release the code for downstream fine-tuning and the complete paper in about ten days. If you have any questions, feel free to discuss.

CarllllWang commented 1 month ago

Thank you for sharing this update! I am very excited about your work and look forward to the opportunity to apply the Visualized BGE to enhance my own projects. I will patiently await the release of your fine-tuning code and the complete paper.  If possible, could you also consider providing a Chinese version of the Visualized BGE, similar to the "bge-visualized-large-zh-v1.5" model? Thank you again for your hard work and generous contribution to the field. I eagerly anticipate your upcoming release!

CarllllWang commented 1 week ago

I hope you're doing great! A while back, we discussed the possibility of the Visualized BGE being made available for fine-tuning and the associated code going open-source, including pre-training (if is needed). You had mentioned an estimated timeline of around ten days for the release.

I'm wondering if there have been any updates or progress on this front. I'm quite eager to explore the Visualized BGE for my ongoing projects, and having access to the code would be extremely helpful.

Thank you for your time, and I appreciate any update you can provide amidst your busy schedule, and I truly appreciate any help you can provide!

JUNJIE99 commented 6 days ago

Thank you for your attention and I apologize for the delay in progress.

Recently, we have released the paper and the Stage-2 training data. The training code might still require some days to be cleaned up.

However, I can initially provide you with the original core training code. If needed, feel free to reach out to zhoujunjie [at] bupt [dot] edu [dot] cn.