Errors when trying to download "yuchenlin/zero-eval" dataset

[lixiaochuan2020]

Hi! Thanks for your amazing work. When I was trying to use mmlu-redux task, I ran this command below: bash zero_eval_local.sh -d mmlu-redux -m TinyLlama/TinyLlama-1.1B-Chat-v1.0 -p tinyllama-chat -s 4 -b 8 > logs/tinyllama-chat.log 2>&1

and got errors:

rank0: Traceback (most recent call last): rank0: File "/home/xiaochul/llm2llm/ZeroEval/src/unified_infer.py", line 117, in rank0: id_strs, chat_history, model_inputs, metadata = load_eval_data(args) rank0: File "/home/xiaochul/llm2llm/ZeroEval/src/unified_utils.py", line 66, in load_eval_data rank0: dataset, id_name = mapping_task_names(data_name) rank0: File "/home/xiaochul/llm2llm/ZeroEval/src/task_configs.py", line 10, in mapping_task_names rank0: dataset = load_dataset("yuchenlin/zero-eval", "mmlu-redux", split="test") rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/datasets/load.py", line 2594, in load_dataset rank0: builder_instance = load_dataset_builder( rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/datasets/load.py", line 2266, in load_dataset_builder rank0: dataset_module = dataset_module_factory( rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/datasets/load.py", line 1914, in dataset_module_factory rank0: raise e1 from None rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/datasets/load.py", line 1834, in dataset_module_factory rank0: dataset_info = hf_api.dataset_info( rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn rank0: return fn(*args, **kwargs) rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 2446, in dataset_info

rank0: File "/home/xiaochul/miniconda3/envs/zeroeval/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 371, in hf_raise_for_status rank0: raise HfHubHTTPError(str(e), response=response) from e rank0: huggingface_hub.utils._errors.HfHubHTTPError: 500 Server Error: Internal Server Error for url: https://huggingface.co/api/datasets/yuchenlin/zero-eval (Request ID: Root=1-66a545ab-22f978203117e18c5f2cd6f1;403ad615-9f7d-4909-b1ba-f0feb40cfa3c) rank0: Internal Error - We're working hard to fix this as soon as possible!

It seems that your dataset "yuchenlin/zero-eval" located on huggingface datasets can not be downloaded. I tried to download it in another separate file like:

dataset = load_dataset("yuchenlin/zero-eval", "mmlu-redux", split="test")

but still failed.

Could you check if there is an error in your dataset? Thanks! :)

[yuchenlin]

this should be an error from the huggingface side. they have some stability issues these days. it should be fixed very soon.