search

xlang-ai / Spider2

Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
https://spider2-sql.github.io
Apache License 2.0
98 stars 6 forks source link

Questions about DailSQL baseline #9

Open kinokopro opened 4 days ago

kinokopro commented 4 days ago

1.According to the readme file, the experimental results were tested on a dataset of 149 records. Could you please specify which 149 records these were and provide the corresponding JSON file? 2.Additionally, regarding the strategies, only [w/Func & w/plan] were mentioned - was external_knowledge used as well?

Thank you for sharing this valuable dataset with the community！

yuxiaooye commented 4 days ago

Hi, Thanks for your issue!

  1. Sorry for not updating the README file sooner. Recently, we completed the evaluation on a version with 319 examples. The JSON file is located at spider2-lite/spider2-lite-319subset.json, and we updated the results in spider2-lite/baselines/dailsql/README.md. Our ultimate goal is to release a version with 500+ examples, and the evaluation results will be updated in the future. Stay tuned!
  2. external_knowledge is used in the vanilla DailSQL. See experimental setting here spider2-lite/baselines/dailsql/README.md.