search

open-compass / T-Eval

[ACL2024] T-Eval: Evaluating Tool Utilization Capability of Large Language Models Step by Step
https://open-compass.github.io/T-Eval/
Apache License 2.0
235 stars 15 forks source link

QWen测试message格式问题 #14

Open gewenbin0992 opened 10 months ago

gewenbin0992 commented 10 months ago

你好，感谢工作！

发现比如reason_str_v1.json数据中经常出现：

  1. 连续的user：[system, user, user]
  2. system穿插在对话中间：[system, user, assistant, system, user]

这种情况，这在QWen中应该是不支持的，请问测试中是如何拼接prompt的？

期待回复。

tonysy commented 10 months ago

You can change the system into user. And merge two user's items into one.