Apart from the issue #78, I've found the following gold query from train_spider.json not being compatible with sqlite :
SELECT document_name FROM documents GROUP BY document_type_code ORDER BY count(*) DESC LIMIT 3 INTERSECT SELECT document_name FROM documents GROUP BY document_structure_code ORDER BY count(*) DESC LIMIT 3
Is this intended? If not, I could open a PR to reorder the ORDER BY clause to come after INTERSECT to solve this.
How to reproduce:
> sqlite3 ./spider/database/document_management/document_management.sqlite
SQLite version 3.39.5 2022-10-14 20:58:05
Enter ".help" for usage hints.
sqlite> SELECT document_name FROM documents GROUP BY document_type_code ORDER BY count(*) DESC LIMIT 3 INTERSECT SELECT document_name FROM documents GROUP BY document_structure_code ORDER BY count(*) DESC LIMIT 3;
Parse error: ORDER BY clause should come after INTERSECT not before
Thanks @taoyds for doing amazing job putting together such an extensive dataset & benchmark for NL-to-SQL models!
Unfortunately I've run into some issues trying to run the official Test Suite Accuracy evaluator on the Spider dataset.
Apart from the issue #78, I've found the following gold query from
train_spider.json
not being compatible with sqlite :Is this intended? If not, I could open a PR to reorder the
ORDER BY
clause to come afterINTERSECT
to solve this.How to reproduce: