search

SeanLee97 / AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard
https://arxiv.org/abs/2309.12871
MIT License
494 stars 33 forks source link

support specify tokenizer_name_or_path #74

Closed SeanLee97 closed 6 months ago

SeanLee97 commented 6 months ago

This feature is to support OpenELM's training. The OpenELM repository does not provide a tokenizer. Instead, it needs to specify the tokenizer of the LLaMA model.