THUDM / LongBench

[ACL 2024] LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
MIT License
659 stars 54 forks source link

Cannot load LongBench-E #20

Closed airaria closed 1 year ago

airaria commented 1 year ago

When I run

from datasets import load_dataset

datasets = ["qasper", "multifieldqa_en", "hotpotqa", "2wikimqa", "gov_report", "multi_news", "trec", \
            "triviaqa", "samsum", "passage_count", "passage_retrieval_en", "lcc", "repobench-p"]

for dataset in datasets:
    data = load_dataset('THUDM/LongBench', f"{dataset}_e", split='test')

the error occurs:

ValueError: BuilderConfig qasper_e not found. Available: ['multifieldqa_en', 'lcc', 'passage_retrieval_zh', 'qasper', 'nq', 'passage_retrieval_en', 'gov_report', 'triviaqa', 'qmsum', 'trec', '2wikimqa', 'dureader', 'lsht', 'passage_count', 'repobench-p', 'hotpotqa', 'narrativeqa', 'vcsum', 'musique', 'multifieldqa_zh']

Is LongBench-E available now ?

bys0318 commented 1 year ago

Thanks for pointing out the bug. It is all fixed now~