search

QwenLM / Qwen2.5-Coder

Qwen2.5-Coder is the code version of Qwen2.5, the large language model series developed by Qwen team, Alibaba Cloud.
3.1k stars 210 forks source link

求 Code 模型在 SQL benchmark 的评估代码 #107

Closed nongfang55 closed 2 weeks ago

nongfang55 commented 2 months ago

看到贵团队放出了部分评估 benchmark 的逻辑，希望参考在 Spider 和 BIRD-SQL 的评估实现。这两个 benchmark 本身在 opencompass 和 harness 都没有集成

cyente commented 2 months ago

We are currently organizing prompts for two SQL-related metrics here. https://github.com/QwenLM/Qwen2.5-Coder/tree/codeqwen1_5/evaluation/text_to_sql

As for the specific evaluation scripts, we are still working ona clean open-source version.

nongfang55 commented 2 months ago

期待 release! // waiting for release!

huybery commented 2 weeks ago

https://github.com/QwenLM/Qwen2.5-Coder/tree/main/qwencoder-eval/instruct/bird-spider