search

defog-ai / sql-eval

Evaluate the accuracy of LLM generated outputs
Apache License 2.0
448 stars 47 forks source link

Expanded correct queries for 3 questions + minor prompt/typo fixes #151

Closed rishsriv closed 1 month ago

rishsriv commented 1 month ago

The 3 questions fixed were:

What is the total number of citations received by each author? Previously, this only relied on sum(publication.citation_num) for the correct answer. Now, COUNT(c.cited) is also accepted as a correct answer

Subtract 2 weeks from the most recent predicted graduation date and give the month. Allow for more date formats

Which papers have the highest number of authors, ordered by the number of authors in descending order? Allow answers that include both the paper id and paper title, instead of just one

wendy-aw commented 1 month ago

Thanks for the fixes! Just two small comments from me