Psycoy / MixEval

The official evaluation suite and dynamic data release for MixEval.
https://mixeval.github.io/
196 stars 28 forks source link

Make System Prompt Arg Available for Local Chat Model Evaluation #18

Closed carstendraschner closed 1 month ago

carstendraschner commented 2 months ago

This PR introduces the ability to pass a system prompt dynamically as part of the evaluation process for local chat models. To achieve this, I've added a new argument to the evaluation process, updated the local chat file and the evaluate file to handle this argument, and added a brief comment to the README file for clarity. Changes:

Thank you for your efforts!