What if I set `enable=false` in `evalscope/registry/config/cfg_single.yaml`?

modelscope / evalscope

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

Apache License 2.0

165 stars 24 forks source link

What if I set `enable=false` in `evalscope/registry/config/cfg_single.yaml`? #106

Open zhimin-z opened 1 month ago

zhimin-z commented 1 month ago

Like reviews_gen or rating_gen? What would happen?

wangxingjun778 commented 1 month ago

The prediction stage will be skipped if enable=false for specific model like chatglm3-6b, in the same way that could be happened in review and rating stages.

The desc of stages: answers_gen: predictions and output answers reviews_gen: to call the judge model like gpt-4 to get review results rating_gen: to compute ELO score