search

modelscope / eval-scope

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
Apache License 2.0
110 stars 14 forks source link

cmmlu评测文件缺失DATASET_ID和SUBJECT_MAPPING等 #26

Closed WSC741606 closed 3 months ago

WSC741606 commented 3 months ago

File "eval-scope/package/llmuses/benchmarks/cmmlu/init.py", line 3, in from llmuses.benchmarks.cmmlu.cmmlu_adapter import DATASET_ID, SUBJECT_MAPPING, SUBSET_LIST, CMMLUAdapter ImportError: cannot import name 'DATASET_ID' from 'llmuses.benchmarks.cmmlu.cmmlu_adapter' (eval-scope/package/llmuses/benchmarks/cmmlu/cmmlu_adapter.py) 看了下mmlu是有这些import项目的，但cmmlu没有，所以加载时缺失报错

wangxingjun778 commented 3 months ago

目前可使用release 0.2分支或master分支来使用cmmlu：

git clone -b release/0.2 https://github.com/modelscope/eval-scope.git

cd eval-scope pip3 install -e .

WSC741606 commented 3 months ago

收到，前几天看到有fix cmmlu的commit就测试了，可以正常跑cmmlu测试，感谢回复！