FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs
MIT License
6.81k stars 491 forks source link

请问如何预训练BGE-M3,旋转编码可以替换为rope吗 #630

Open BrownSweater opened 5 months ago

staoxiao commented 5 months ago

请参考:https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain#3-train 可以替换为rope,但需要修改模型代码。