search

lanyunshi / ConversationalKBQA

15 stars 2 forks source link

ConvCSQA #4

Open SRL94 opened 2 years ago

SRL94 commented 2 years ago

Hi, I am trying to reproduce the convCSQA dataset but I have one question about it. Could you please answer it?

If the question type description did not explicitly mention that this question contains an "indirect" subject, we removed it.

What does it mean? Could you please give an example?

SRL94 commented 2 years ago

Also, does ConvCSQA only contain simple questions? Excluding comparative, logical, quantitative, verification questions?

iCloud112 commented 2 years ago

Have you reproduced the ConvCSQA dataset?

SRL94 commented 2 years ago

Hi, I reproduced it based on the instructions on the paper. However, the statistics did not match the paper. Therefore, I did not use this dataset in my research.

zeaplau commented 2 years ago

I also reproduced this dataset with the help of the author (Prof. Lan sent me an email for preprocessing), but the statistics also did not match. The reason that the statistics did not match in paper may be the dataset used in the paper is CSQA_v1 but the CSQA_v9 is provided I guessed. However, the reproduced result is relatively close ... 🤔. And I also received the code that the author preprocess this dataset, which is just after NER (cost a lot of time), and reformulated the format similar to CONVEX. Here is the URL of the CSQA dataset that just after NER processing, the ConvCSQA can be created following the steps in the paper: https://drive.google.com/file/d/1KbVQVZHgoMY_mLwOrg2rUfKBsu4OkcOa/view?usp=share_link

SRL94 commented 2 years ago

Thank you very much! Yes the reproduced results are relatively close.

iCloud112 commented 2 years ago

Hi, I reproduced it based on the instructions on the paper. However, the statistics did not match the paper. Therefore, I did not use this dataset in my research.

Thank you for your reply. And how about the ConvQuestions dataset? Did your results match the paper reported or not?

iCloud112 commented 2 years ago

I also reproduced this dataset with the help of the author (Prof. Lan sent me an email for preprocessing), but the statistics also did not match. The reason that the statistics did not match in paper may be the dataset used in the paper is CSQA_v1 but the CSQA_v9 is provided I guessed.

Thank you very much! The reproduced result is relatively close. Does this statement mean that the results with different question types are close?