run harness on A770 error

[tao-ov]

when i run harness as the following link on A770

https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/dev/benchmark/harness/run_llb.py

the cmd is：python run_llb.py --model ipex-llm --pretrained /home/test/models/LLM/baichuan2-7b/pytorch/ --precision sym_int4 --device xpu --tasks hellaswag --batch 1 --no_cache

it occurs this error： RuntimeError: Job config of task=hellaswag, precision=sym_int4 failed. Error Message: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte

[lalalapotter]

Could you please remove the try-except clause here and provide more error log?

[tao-ov]

(llm) test@test-Z590-VISION-D:~/ipexllm_whowhat/ipex-llm/python/llm/dev/benchmark/harness$ python run_llb.py --model ipex-llm --pretrained /home/test/models/LLM/baichuan2-7b/pytorch/ --precision sym_int4 --device xpu --tasks hellaswag --batch 1 --no_cache /home/test/miniforge3/envs/llm/lib/python3.11/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations warnings.warn( /home/test/miniforge3/envs/llm/lib/python3.11/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( 2024-10-30 11:06:38,081 - INFO - intel_extension_for_pytorch auto imported Selected Tasks: ['hellaswag'] The repository for /home/test/models/LLM/baichuan2-7b/pytorch/ contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//home/test/models/LLM/baichuan2-7b/pytorch/. You can avoid this prompt in future by passing the argument trust_remote_code=True.

Do you wish to run the custom code? [y/N] y 2024-10-30 11:06:40,365 - WARNING - Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers pip install xformers. /home/test/miniforge3/envs/llm/lib/python3.11/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.get(instance, owner)() 2024-10-30 11:06:55,197 - INFO - Converting the current model to sym_int4 format...... Traceback (most recent call last): File "/home/test/ipexllm_whowhat/ipex-llm/python/llm/dev/benchmark/harness/run_llb.py", line 147, in main() File "/home/test/ipexllm_whowhat/ipex-llm/python/llm/dev/benchmark/harness/run_llb.py", line 101, in main results = evaluator.simple_evaluate( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/lm-evaluation-harness/lm_eval/utils.py", line 243, in _wrapper return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/test/lm-evaluation-harness/lm_eval/evaluator.py", line 89, in simple_evaluate task_dict = lm_eval.tasks.get_task_dict(tasks) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/lm-evaluation-harness/lm_eval/tasks/init.py", line 390, in get_task_dict task_name_dict = { ^ File "/home/test/lm-evaluation-harness/lm_eval/tasks/init.py", line 391, in task_name: get_task(task_name)() ^^^^^^^^^^^^^^^^^^^^^ File "/home/test/lm-evaluation-harness/lm_eval/base.py", line 481, in init self.download(data_dir, cache_dir, download_mode) File "/home/test/lm-evaluation-harness/lm_eval/base.py", line 510, in download self.dataset = datasets.load_dataset( ^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/datasets/load.py", line 2606, in load_dataset builder_instance = load_dataset_builder( ^^^^^^^^^^^^^^^^^^^^^ File "/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/datasets/load.py", line 2277, in load_dataset_builder dataset_module = dataset_module_factory( ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/datasets/load.py", line 1923, in dataset_module_factory raise e1 from None File "/home/test/miniforge3/envs/llm/lib/python3.11/site-packages/datasets/load.py", line 1875, in dataset_module_factory can_load_config_from_parquet_export = "DEFAULT_CONFIG_NAME" not in f.read() ^^^^^^^^ File "", line 322, in decode UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte