search

MohammadrezaPourreza / Few-shot-NL2SQL-with-prompting

MIT License
305 stars 61 forks source link

Cost Estimates #18

Open guyonhisowntrip opened 11 months ago

guyonhisowntrip commented 11 months ago

Hey great paper. Curious to know if you have done cost estimates on running a query? How does the cost vary between easy, non nested and nested queries? My assumption is that for answering one question, we hit LLM 4 times, is my understanding correct?

MohammadrezaPourreza commented 11 months ago

Hi thanks for your interesting in our work, number of tokens used for Non-nested and nested queries is larger and the cost per question on Spider dataset would be around 0.7$ using GPT-4 and for easy class is around 0.5$ per question using GPT-4. Yes, there are 4 LLM calls per question.

guyonhisowntrip commented 11 months ago

Great thanks for your reply. What is the average time per query (easy, nested and non nested) to get the answer? I am asking these questions, as building a solution for enterprise will take in these considerations