ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.

hyperonym / basaran

Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.

MIT License

1.29k stars 80 forks source link

ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported. #139

Closed josephrocca closed 1 year ago

josephrocca commented 1 year ago

Running this command:

docker run -p 80:80 -e MODEL=decapoda-research/llama-7b-hf hyperonym/basaran:0.15.3

Gives:

ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.

It worked fine with the cerebras/Cerebras-GPT-1.3B and bigscience/bloom-560m models.

(Thanks for your work on this project! Currently integrating it with OpenCharacters)

peakji commented 1 year ago

Hi @josephrocca, decapoda-research/llama-7b-hf hasn't been updated to use the new class name (LLaMATokenizer -> LlamaTokenizer), which was later finalized in HF Transformers v4.28.0. You could just pick another LLaMA model from Hugging Face Hub.