Closed openvino-book closed 1 month ago
@openvino-book according to this issue on hf hub, you need to upgrade transformers for loading llama3.1 https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/discussions/15
@openvino-book according to this issue on hf hub, you need to upgrade transformers for loading llama3.1 https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/discussions/15
Thank you @eaidova , it works!
pip install --upgrade transformers
the worked version of transformers is 4.43.3
Hi, @eaidova , after update the transformers, the convert.py works well.
However, the benchmark.py raise an error as below when running the converted Llama3.1 IR model.
running on the CPU has the same error
Hi, @eaidova very interesting, there are not openvino_tokenizer.xml & openvino_detokenizer.xml in the converted model D:\openvino.genai\llm_bench\python\llama31_ov\pytorch\dldt\FP16
Hi, @eaidova very interesting, there are not openvino_tokenizer.xml & openvino_detokenizer.xml in the converted model D:\openvino.genai\llm_bench\python\llama31_ov\pytorch\dldt\FP16
@openvino-book If this issue can still be reproduced with latest openvino.genai? https://github.com/openvinotoolkit/openvino.genai
Run convert.py on Meta-Llama-3.1-8B-Instruct model, Error occur as below: