search

xlang-ai / Spider2

Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
https://spider2-sql.github.io
Apache License 2.0
82 stars 2 forks source link

The amount of functions is different in spider2-lite.json and the corresponding gold SQL query #2

Open riddiculous opened 5 days ago

riddiculous commented 5 days ago

For example, in bq026, the functions claimed in spider2-lite.json are: "special_function": [ "aggregate-functions/ANY_VALUE", "aggregate-functions/ARRAY_AGG", "aggregate-functions/GROUPING", "aggregate-functions/STRING_AGG", "conversion-functions/CAST", "mathematical-functions/FLOOR", "string-functions/REGEXP_CONTAINS", "conditional-functions/IF", "other-functions/UNNEST", "other-functions/ARRAY_SUBSCRIPT" ] However, the function GROUPING and IF are not used in the gold SQL of bq026. I wonder whether they need to be consistent.

lfy79001 commented 5 days ago

Hi, the "special_function" is intended solely for analysis; it is not a fundamental setting of our task nor the only ground truth. In fact, there may be multiple gold SQL queries that can solve a task, each utilizing different functions. Regarding your question, we acknowledge this oversight and will correct the bug promptly.