search

salesforce / WikiSQL

A large annotated semantic parsing corpus for developing natural language interfaces.
BSD 3-Clause "New" or "Revised" License
1.6k stars 320 forks source link

How to evaluate a Seq2Seq model ? #78

Open mellahysf opened 3 years ago

mellahysf commented 3 years ago

Hi,

I have built a se2seq model that takes the question and generates the SQL query directly.

My question is how to evaluate my model since the predictions are in sequence format ("select .... from ... where...")?

It's very urgent, please.

Thanks

jaytimbadia commented 3 years ago

Hi @mellahysf How can one train on a custom SQLite file or database? Can you please provide the steps or can share your training script if possible?

Thank you!

Timothyxxx commented 2 years ago

Hi~A simple solution is to use the evaluation from Spider dataset https://github.com/taoyds/spider which has great written parser to formulate the sql. Otherwise, since the formulation of WikiSQL is quiet simple, you can just write a parser for you generated SQL.