AlibabaResearch / DAMO-ConvAI

DAMO-ConvAI: The official repository which contains the codebase for Alibaba DAMO Conversational AI.
MIT License
1.1k stars 178 forks source link

Possible mistake in dev dataset #96

Closed josem7 closed 4 months ago

josem7 commented 8 months ago
{'question_id': 21,
 'db_id': 'california_schools',
 'question': 'In Los Angeles how many schools have more than 500 free meals but less than 700 free or reduced price meals for K-12?',
 'evidence': '',
 'SQL': "SELECT COUNT(CDSCode) FROM frpm WHERE `County Name` = 'Los Angeles' AND `Free Meal Count (K-12)` > 500 AND `Free Meal Count (K-12)` < 700",
 'difficulty': 'simple',
 'prediciton': "SELECT COUNT(T1.School) FROM frpm AS T1 INNER JOIN schools AS T2 ON T1.CDSCode = T2.CDSCode WHERE T2.City = 'Los Angeles' AND T1.`Free Meal Count (K-12)` > 500 AND T1.`FRPM Count (K-12)` < 700",
 'result': 0}

In this question, it is asked for "less than 700 free or reduced price meals for K-12".

Looking at the description of FRPM Count (K-12) it says as follows: "Free or Reduced Price Meal Count (K-12)"

So the second part of the query should be FRPM Count (K-12) < 700

huybery commented 8 months ago

Could u contribute it in https://github.com/AlibabaResearch/DAMO-ConvAI/issues/39? We will revise it in next version. Thanks a lot!

CastFX commented 5 months ago

https://arxiv.org/pdf/2402.12243v2.pdf

accpatrick commented 5 months ago

@josem7 Thanks a lot for your feedback and we will revise this in next version.

accpatrick commented 5 months ago

@CastFX Thanks for letting us know this. Please refer to this (https://github.com/AlibabaResearch/DAMO-ConvAI/issues/39#issuecomment-1969316367) for better understanding. Thanks.