search

meta-llama / llama-models

Utilities intended for use with Llama models.
Other
4.88k stars 838 forks source link

Make `tokenizer_path` optional #182

Closed ashwinb closed 1 month ago

ashwinb commented 1 month ago

We have Tokenizer.get_instance() which looks up the default path using the baked in tokenizer.model in the repository anyway.