convert.py do NOT support Meta-Llama-3.1-8B-Instruct

[openvino-book]

Run convert.py on Meta-Llama-3.1-8B-Instruct model, Error occur as below:

ValueError: rope_scaling must be a dictionary with two fields, type and factor, got {'factor': 8.0, 'low_freq_factor': 1.0, 'high_freq_factor': 4.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}

[eaidova]

@openvino-book according to this issue on hf hub, you need to upgrade transformers for loading llama3.1 https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/discussions/15

[openvino-book]

@openvino-book according to this issue on hf hub, you need to upgrade transformers for loading llama3.1 https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/discussions/15

Thank you @eaidova , it works!

pip install --upgrade transformers

the worked version of transformers is 4.43.3

[openvino-book]

Hi, @eaidova , after update the transformers, the convert.py works well.

However, the benchmark.py raise an error as below when running the converted Llama3.1 IR model.

[openvino-book]

running on the CPU has the same error

[openvino-book]

Hi, @eaidova very interesting, there are not openvino_tokenizer.xml & openvino_detokenizer.xml in the converted model D:\openvino.genai\llm_bench\python\llama31_ov\pytorch\dldt\FP16

[peterchen-intel]

Hi, @eaidova very interesting, there are not openvino_tokenizer.xml & openvino_detokenizer.xml in the converted model D:\openvino.genai\llm_bench\python\llama31_ov\pytorch\dldt\FP16

@openvino-book If this issue can still be reproduced with latest openvino.genai? https://github.com/openvinotoolkit/openvino.genai