search

SeanLee97 / AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard
https://arxiv.org/abs/2309.12871
MIT License
397 stars 30 forks source link

support specify tokenizer_name_or_path #74

Closed SeanLee97 closed 1 month ago

SeanLee97 commented 1 month ago

This feature is to support OpenELM's training. The OpenELM repository does not provide a tokenizer. Instead, it needs to specify the tokenizer of the LLaMA model.