Closed Acemid closed 6 months ago
Please offer your error logs, in your workdir/time/logs/infer or workdir/time/logs/eval
python run.py --datasets ceval_gen --hf-path internlm2-chat-1_8b --tokenizer-path internlm2-chat-1_8b --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug
results:
signal.SIGALRM is not available on this platform
signal.SIGALRM is not available on this platform
04/29 00:42:54 - OpenCompass - ERROR - D:\NingZhangZJU\opencompass\opencompass\tasks\openicl_eval.py - _score - 241 -
Task [opencompass.models.huggingface.HuggingFace_D:\NingZhangZJU\opencompass\internlm2-chat-1_8b/ceval-college_programming]: No predictions found.
04/29 00:42:54 - OpenCompass - INFO - time elapsed: 3.39s
04/29 00:42:55 - OpenCompass - DEBUG - Get class OpenICLEvalTask
from "task" registry in "opencompass"
04/29 00:42:55 - OpenCompass - DEBUG - An OpenICLEvalTask
instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
signal.SIGALRM is not available on this platform
signal.SIGALRM is not available on this platform
04/29 00:43:03 - OpenCompass - ERROR - D:\NingZhangZJU\opencompass\opencompass\tasks\openicl_eval.py - _score - 241 - Task [opencompass.models.huggingface.HuggingFace_D:\NingZhangZJU\opencompass\internlm2-chat-1_8b/ceval-college_physics]: No predictions found.
04/29 00:43:03 - OpenCompass - INFO - time elapsed: 3.54s
Are you running on a windows platform? it seems it is caused by 'signal.SIGALRM is not available on this platform', and opencompass is not well compatible with windows, we suggest to run on linux
fine~
先决条件
问题类型
我正在使用官方支持的任务/模型/数据集进行评估。
环境
python run.py --models hf_internlm2_1_8b --datasets mmlu_ppl
重现问题 - 代码/配置示例
dataset version metric mode internlm2-1.8b-hf
lukaemon_mmlu_college_biology - - - - lukaemon_mmlu_college_chemistry - - - - lukaemon_mmlu_college_computer_science - - - - lukaemon_mmlu_college_mathematics - - - - lukaemon_mmlu_college_physics - - - - lukaemon_mmlu_electrical_engineering - - - - lukaemon_mmlu_astronomy - - - - lukaemon_mmlu_anatomy - - - - lukaemon_mmlu_abstract_algebra - - - - lukaemon_mmlu_machine_learning - - - - lukaemon_mmlu_clinical_knowledge - - - - lukaemon_mmlu_global_facts - - - - lukaemon_mmlu_management - - - - lukaemon_mmlu_nutrition - - - - lukaemon_mmlu_marketing - - - - lukaemon_mmlu_professional_accounting - - - - lukaemon_mmlu_high_school_geography - - - - lukaemon_mmlu_international_law - - - - lukaemon_mmlu_moral_scenarios - - - - lukaemon_mmlu_computer_security - - - - lukaemon_mmlu_high_school_microeconomics - - - - lukaemon_mmlu_professional_law - - - - lukaemon_mmlu_medical_genetics - - - - lukaemon_mmlu_professional_psychology - - - - lukaemon_mmlu_jurisprudence - - - - lukaemon_mmlu_world_religions - - - - lukaemon_mmlu_philosophy - - - - lukaemon_mmlu_virology - - - - lukaemon_mmlu_high_school_chemistry - - - - lukaemon_mmlu_public_relations - - - - lukaemon_mmlu_high_school_macroeconomics - - - - lukaemon_mmlu_human_sexuality - - - - lukaemon_mmlu_elementary_mathematics - - - - lukaemon_mmlu_high_school_physics - - - - lukaemon_mmlu_high_school_computer_science - - - - lukaemon_mmlu_high_school_european_history - - - - lukaemon_mmlu_business_ethics - - - - lukaemon_mmlu_moral_disputes - - - - lukaemon_mmlu_high_school_statistics - - - - lukaemon_mmlu_miscellaneous - - - - lukaemon_mmlu_formal_logic - - - - lukaemon_mmlu_high_school_government_and_politics - - - - lukaemon_mmlu_prehistory - - - - lukaemon_mmlu_security_studies - - - - lukaemon_mmlu_high_school_biology - - - - lukaemon_mmlu_logical_fallacies - - - - lukaemon_mmlu_high_school_world_history - - - - lukaemon_mmlu_professional_medicine - - - - lukaemon_mmlu_high_school_mathematics - - - - lukaemon_mmlu_college_medicine - - - - lukaemon_mmlu_high_school_us_history - - - - lukaemon_mmlu_sociology - - - - lukaemon_mmlu_econometrics - - - - lukaemon_mmlu_high_school_psychology - - - - lukaemon_mmlu_human_aging - - - - lukaemon_mmlu_us_foreign_policy - - - - lukaemon_mmlu_conceptual_physics - - - -
重现问题 - 命令或脚本
相同的环境,使用笔记本3050平台。
重现问题 - 错误信息
结果没有正常显示模型得分
其他信息
、