Evaluation：UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

[manlenzzz]

Thank you very much for doing such great open-source work!

i try: CUDA_VISIBLE_DEVICES=X bash scripts/evaluate.sh PATH_OR_NAME_TO_BASE_MODEL PATH_TO_SAVE_TUNE_MODEL PATH_TO_PRUNE_MODEL EPOCHS_YOU_WANT_TO_EVALUATE

but get the result: Selected Tasks: ['piqa', 'boolq', 'arc_challenge', 'hellaswag', 'openbookqa', 'arc_easy', 'winogrande'] Load from Pruned Model: prune_log/llama_prune/pytorch_model.bin Config: LoraConfig(peft_type='LORA', base_model_name_or_path='baffo32/decapoda-research-llama-7B-hf', task_type='CAUSAL_LM', inference_mode=True, r=8, target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'down_proj', 'up_proj'], lora_alpha=16, lora_dropout=0.05, fan_in_fan_out=False, bias='none', modules_to_save=None, init_lora_weights=True) Load from adapter: adapter_model.bin Traceback (most recent call last): File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/main.py", line 115, in main() File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/main.py", line 81, in main results = evaluator.simple_evaluate( File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/utils.py", line 182, in _wrapper return fn(*args, **kwargs) File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/evaluator.py", line 81, in simple_evaluate task_dict = lm_eval.tasks.get_task_dict(tasks) File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/tasks/init.py", line 344, in get_task_dict task_name_dict = { File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/tasks/init.py", line 345, in task_name: get_task(task_name)() File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/base.py", line 456, in init self.download(data_dir, cache_dir, download_mode) File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/base.py", line 485, in download self.dataset = datasets.load_dataset( File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 2587, in load_dataset builder_instance = load_dataset_builder( File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 2259, in load_dataset_builder dataset_module = dataset_module_factory( File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 1910, in dataset_module_factory raise e1 from None File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 1862, in dataset_module_factory can_load_config_from_parquet_export = "DEFAULT_CONFIG_NAME" not in f.read() File "/usr/lib/python3.10/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

[BrownTan]

I think you didn't use a magic.