issues
search
modelscope
/
eval-scope
A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
Apache License 2.0
101
stars
12
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add en readme
#70
wangxingjun778
closed
3 days ago
0
perf 测试不输出结果
#69
hetian127
opened
4 days ago
7
enhance compatibility with Windows system.
#68
ChunchunWang5
closed
49 minutes ago
0
运行run.py出错，说没有eval_config
#67
aoligei178
opened
5 days ago
2
feat: add openai/eas sdk logits support.
#66
Chen9154
closed
4 days ago
0
使用llmuses perf 命令报错
#65
hetian127
closed
4 days ago
2
perf流式输出报错
#64
ccly1996
opened
6 days ago
4
fix dashscope api bug
#63
liuyhwangyh
closed
5 days ago
0
Add opencompass as evaluation backend
#62
wangxingjun778
closed
5 days ago
0
简介里写着 ：统一model接入，兼容多个系列模型的generate、chat接口 ，我看源码里写的是to do ,是还没支持吗
#61
meichangsu1
closed
1 week ago
1
Add custom request field
#60
liuyhwangyh
opened
2 weeks ago
0
Update toolbench
#59
wangxingjun778
closed
3 weeks ago
0
Add toolbench_static benchmark
#58
wangxingjun778
closed
3 weeks ago
0
命名的demo英文名翻车了，嘿嘿
#57
betasspace
closed
3 weeks ago
3
update gsm8k yaml
#56
wangxingjun778
closed
4 weeks ago
0
Del wandb and streamlit in requirements
#55
wangxingjun778
closed
4 weeks ago
0
Set default few_shot to 0 for gsm8k task yaml
#54
wangxingjun778
closed
4 weeks ago
0
update openai requirement
#53
wangxingjun778
closed
1 month ago
0
Support batch prediction for custom model
#52
wangxingjun778
closed
1 month ago
0
add custom parameters
#51
liuyhwangyh
closed
3 weeks ago
0
Fix/arena temlate type
#50
wangxingjun778
closed
1 month ago
0
Fix/arena temlate type
#49
wangxingjun778
closed
1 month ago
0
微调模型评测&自建评测数据集
#48
HaltonJiang
opened
1 month ago
0
fix: update ChatGenerationModelAdapter get strategy.
#47
Chen9154
closed
1 month ago
0
dataset and api process separate
#46
liuyhwangyh
closed
1 month ago
0
Refine fuzzy_match in template file
#45
wangxingjun778
closed
1 month ago
0
怎么加载本地模型呀
#44
FrankGnor
opened
1 month ago
0
Dev/refactor 0.2
#43
wangxingjun778
closed
2 months ago
0
Dev/refactor 0.2
#42
wangxingjun778
closed
2 months ago
0
Optimize perf benchmark
#41
liuyhwangyh
closed
2 months ago
0
feat: update data/model/general_qa adapter & add eval&infer engine.
#40
Chen9154
closed
2 months ago
0
提供的本地数据集文件夹结构有误，无法直接使用
#39
thewangcj
opened
2 months ago
3
Merge main to release/0.3
#38
wangxingjun778
closed
2 months ago
0
Update generation template
#37
wangxingjun778
closed
2 months ago
0
Fix cmmlu and support swift config
#36
wangxingjun778
closed
2 months ago
0
Remove swift dependencies
#35
wangxingjun778
closed
2 months ago
0
Add generation config template for model chat interface
#34
wangxingjun778
closed
2 months ago
0
不能加载本地模型
#33
v-yunbin
opened
2 months ago
0
Support custom config and swift evaluation
#32
wangxingjun778
closed
2 months ago
0
Support loading cmmlu from local disk
#31
wangxingjun778
closed
3 months ago
0
Fix/general qa load
#30
wangxingjun778
closed
3 months ago
0
fix cmmlu bugs.
#29
Chen9154
closed
3 months ago
0
Dev/fix local eval
#28
wangxingjun778
closed
3 months ago
0
Fix local eval
#27
wangxingjun778
closed
3 months ago
0
cmmlu评测文件缺失DATASET_ID和SUBJECT_MAPPING等
#26
WSC741606
closed
3 months ago
2
请教一下自定义模型和数据集评测如何实现
#25
WSC741606
closed
3 months ago
4
model_id为local ckpt路径时，infer template映射和解析出现问题
#24
wangxingjun778
opened
3 months ago
0
Fix run script
#23
wangxingjun778
closed
3 months ago
0
feat: add CMMLU support
#22
Chen9154
closed
3 months ago
0
关于few-shot的一些问题
#21
MrZhang1996
opened
3 months ago
3
Next