search

whoiskatrin / sql-translator

SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.
https://www.sqltranslate.app/
MIT License
4.16k stars 356 forks source link

In order to increase adaptability, this project needs to introduce industrialized test cases. #48

Closed jack-kwok80 closed 1 year ago

jack-kwok80 commented 1 year ago

This is a very interesting project. Currently, the functions are relatively simple. I think we need to introduce industrialized test cases to help push it forward. What do you think?

whoiskatrin commented 1 year ago

I think it's a great suggestion. Do you have something concrete in mind?

jack-kwok80 commented 1 year ago

Yes, I have some suggestion as follows. As the default schema of this project is “cars”. I think we can simulate a car dealership to establish a basic data model, consisting of a car table, a customer table, and a sales fact table. The sales table records the purchase history of users, including purchase time, price, quantity, and car model. Based on these three tables, various joint queries can be performed, such as querying sales by time, identifying popular car models during a certain period, etc. In this way, we can test the effectiveness of SQL generation based on an actual problem. As for the testing method, we can first write the test SQL, use ChatGPT to generate natural language, manually organize it to create a test case combining natural language and SQL. Finally, we can establish an automatic testing script and run these cases every time when release an official version to ensure project quality. Do you think there are any problems? If it's ok, I can add these cases.

whoiskatrin commented 1 year ago

@weizhan yes, go for a PR

jack-kwok80 commented 1 year ago

I have created some test cases using ava, and I need add "type":"module" to the package.json to run those cases. But I got error as follows when run "npm run build" command, I'm freshman to typescript, do you know how to fix it? @whoiskatrin

Thanks

Project: https://github.com/weizhan/sql-translator

ERROR INFO: ReferenceError: module is not defined in ES module scope This file is being treated as an ES module because it has a '.js' file extension and 'F:\bigdata\sql-translator\package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.

whoiskatrin commented 1 year ago

@jack-kwok80 you are trying to use a CommonJS-style import in a JavaScript file that's being treated as an ES module due to the package.json configuration; the module object is not defined in the ES module scope

jack-kwok80 commented 1 year ago

I change the test framework to jest, and it works now. I have request a PR.

but some cases failed, there are some limits with model text-davinci-003, we can figure out later.