Closed jerome-white closed 2 weeks ago
Potentially related:
Hi @jerome-white, thnaks for reporting.
However, I cannot reproduce your issue:
>>> from datasets import get_dataset_config_names
>>> get_dataset_config_names("open-llm-leaderboard/details_davidkim205__Rhea-72b-v0.5")
['harness_arc_challenge_25',
'harness_gsm8k_5',
'harness_hellaswag_10',
'harness_hendrycksTest_5',
'harness_hendrycksTest_abstract_algebra_5',
'harness_hendrycksTest_anatomy_5',
'harness_hendrycksTest_astronomy_5',
'harness_hendrycksTest_business_ethics_5',
'harness_hendrycksTest_clinical_knowledge_5',
'harness_hendrycksTest_college_biology_5',
'harness_hendrycksTest_college_chemistry_5',
'harness_hendrycksTest_college_computer_science_5',
'harness_hendrycksTest_college_mathematics_5',
'harness_hendrycksTest_college_medicine_5',
'harness_hendrycksTest_college_physics_5',
'harness_hendrycksTest_computer_security_5',
'harness_hendrycksTest_conceptual_physics_5',
'harness_hendrycksTest_econometrics_5',
'harness_hendrycksTest_electrical_engineering_5',
'harness_hendrycksTest_elementary_mathematics_5',
'harness_hendrycksTest_formal_logic_5',
'harness_hendrycksTest_global_facts_5',
'harness_hendrycksTest_high_school_biology_5',
'harness_hendrycksTest_high_school_chemistry_5',
'harness_hendrycksTest_high_school_computer_science_5',
'harness_hendrycksTest_high_school_european_history_5',
'harness_hendrycksTest_high_school_geography_5',
'harness_hendrycksTest_high_school_government_and_politics_5',
'harness_hendrycksTest_high_school_macroeconomics_5',
'harness_hendrycksTest_high_school_mathematics_5',
'harness_hendrycksTest_high_school_microeconomics_5',
'harness_hendrycksTest_high_school_physics_5',
'harness_hendrycksTest_high_school_psychology_5',
'harness_hendrycksTest_high_school_statistics_5',
'harness_hendrycksTest_high_school_us_history_5',
'harness_hendrycksTest_high_school_world_history_5',
'harness_hendrycksTest_human_aging_5',
'harness_hendrycksTest_human_sexuality_5',
'harness_hendrycksTest_international_law_5',
'harness_hendrycksTest_jurisprudence_5',
'harness_hendrycksTest_logical_fallacies_5',
'harness_hendrycksTest_machine_learning_5',
'harness_hendrycksTest_management_5',
'harness_hendrycksTest_marketing_5',
'harness_hendrycksTest_medical_genetics_5',
'harness_hendrycksTest_miscellaneous_5',
'harness_hendrycksTest_moral_disputes_5',
'harness_hendrycksTest_moral_scenarios_5',
'harness_hendrycksTest_nutrition_5',
'harness_hendrycksTest_philosophy_5',
'harness_hendrycksTest_prehistory_5',
'harness_hendrycksTest_professional_accounting_5',
'harness_hendrycksTest_professional_law_5',
'harness_hendrycksTest_professional_medicine_5',
'harness_hendrycksTest_professional_psychology_5',
'harness_hendrycksTest_public_relations_5',
'harness_hendrycksTest_security_studies_5',
'harness_hendrycksTest_sociology_5',
'harness_hendrycksTest_us_foreign_policy_5',
'harness_hendrycksTest_virology_5',
'harness_hendrycksTest_world_religions_5',
'harness_truthfulqa_mc_0',
'harness_winogrande_5',
'results']
Maybe it was just a temporary issue...
Maybe it was just a temporary issue...
Perhaps. I've changed my workflow to use the hub's HfFileSystem
, so for now this is no longer a blocker for me. I'll reopen the issue if that changes.
Describe the bug
When trying to get config names or load any dataset within the open-llm-leaderboard ecosystem (
open-llm-leaderboard/details_
) I receive the DataFilesNotFoundError. For the last month or so I've been loading datasets from the leaderboard almost everyday; yesterday was the first time I started seeing this.Steps to reproduce the bug
This snippet has three cells:
I've chosen "davidkim205"'s Rhea-72b-v0.5 model because it is one of the best performers on the leaderboard should likely have no dataset issues:
Expected behavior
No exceptions from
get_dataset_config_names
orload_dataset
Environment info
datasets
version: 2.19.0huggingface_hub
version: 0.23.0fsspec
version: 2024.3.1