ValueError: Tokenizer class LlamaTokenizer does not exist or is not currently imported. Getting this error when I try to run the below code:

[SharathK-Tiger]

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NousResearch/Nous-Hermes-Llama2-13b")
model = AutoModelForCausalLM.from_pretrained("NousResearch/Nous-Hermes-Llama2-13b")

I'm trying to use a Llama-2 model for table summarization purposes. I've tried the other Llama 2 models as well and haven't had any success. This is all being done in a databricks notebook and there's more than enough GPU memory. I've also ensured that all required libraries are up to date (transformers, sentencepiece, accelerate etc.)

[ArthurZucker]

Hey! This hast nothing to do with tokenizers and is more related to the way you installed transformers! Make sure you are using the latest version 🤗 I'll close this but feel free to open an issue on transformers if you are not able to run your script. It works perfectly for me.

>>> import transformers
>>> print(transformers.__version__)