是否支持多卡推理或LORA？

SeanLee97 / AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard

https://arxiv.org/abs/2309.12871

MIT License

454 stars 32 forks source link

Closed Chineselock closed 10 months ago

Chineselock commented 10 months ago

您好，能否调整API使模型推理时存储在多张显卡上？我现在有多张24G显存的显卡并且我希望能够运行LLama-7B进行embedding