Open tao-ov opened 1 day ago
Could you please remove the try-except clause here and provide more error log?
(llm) test@test-Z590-VISION-D:~/ipexllm_whowhat/ipex-llm/python/llm/dev/benchmark/harness$ python run_llb.py --model ipex-llm --pretrained /home/test/models/LLM/baichuan2-7b/pytorch/ --precision sym_int4 --device xpu --tasks hellaswag --batch 1 --no_cache
/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io
, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg
or libpng
installed before building torchvision
from source?
warn(
2024-10-30 11:06:38,081 - INFO - intel_extension_for_pytorch auto imported
Selected Tasks: ['hellaswag']
The repository for /home/test/models/LLM/baichuan2-7b/pytorch/ contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//home/test/models/LLM/baichuan2-7b/pytorch/.
You can avoid this prompt in future by passing the argument trust_remote_code=True
.
Do you wish to run the custom code? [y/N] y
2024-10-30 11:06:40,365 - WARNING - Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers
pip install xformers.
/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
return self.fget.get(instance, owner)()
2024-10-30 11:06:55,197 - INFO - Converting the current model to sym_int4 format......
Traceback (most recent call last):
File "/home/test/ipexllm_whowhat/ipex-llm/python/llm/dev/benchmark/harness/run_llb.py", line 147, in
when i run harness as the following link on A770
https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/dev/benchmark/harness/run_llb.py
the cmd is:python run_llb.py --model ipex-llm --pretrained /home/test/models/LLM/baichuan2-7b/pytorch/ --precision sym_int4 --device xpu --tasks hellaswag --batch 1 --no_cache
it occurs this error: RuntimeError: Job config of task=hellaswag, precision=sym_int4 failed. Error Message: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte