allenai / natural-instructions

Expanding natural instructions
https://instructions.apps.allenai.org/
Apache License 2.0
949 stars 188 forks source link

Repeated instances found in the tasks #39

Closed danyaljj closed 3 years ago

danyaljj commented 3 years ago

Please send a PR with the fix.

danyaljj commented 3 years ago

@shailaja183 your task also has repeated instances.

Check for Passage: Sandra journeyed to the bathroom. Daniel moved to the garden. Question: Where is Daniel? in the following task: task083_babi_t1_single_supporting_fact_answer_generation

danyaljj commented 3 years ago

@kurbster also see the following repeated line:

 {'input': '[14, -18]', 'output': ['[13, -20]']}

in task096_conala_list_index_subtraction.json

danyaljj commented 3 years ago

@shailaja183 Also see

 {'input': 'Passage: Fact 1- John went to the kitchen. Fact 2- John went to the hallway. Question: Where is John? Answer: hallway', 'output': ['Fact 2']}

in: task084_babi_t1_single_supporting_fact_identify_relevant_fact.json

danyaljj commented 3 years ago

@kurbster See

 {'input': '[0, 2, 4, 4, 1, 5]', 'output': ['[0, 2, 1, 5]']}

in task097_conala_remove_duplicates.json

danyaljj commented 3 years ago

@kurbster and

 {'input': "['H', 'T']", 'output': ['HT']}

in

task079_conala_concat_strings.json
danyaljj commented 3 years ago

@shailaja183

 {'input': 'Passage: John went to the kitchen. John went to the hallway.', 'output': ['Where is John?']}

in

task082_babi_t1_single_supporting_fact_question_generation.json
danyaljj commented 3 years ago

@shailaja183 @kurbster just checking if you have seen my earlier comments.

swarooprm commented 3 years ago

@kurbster , also when you fix these rename task075 to the latest task number (107 at this point), task075 is repeated with another submitted task

shailaja183 commented 3 years ago

I have fixed babi tasks regarding repetition of instances. Let me know if the issue persists on my end. New commit: repeat instances fix - fa802e9

danyaljj commented 3 years ago

@shailaja183 you gotta put that commit on a branch and send a PR (so that we can merge it with our main branch).

kurbster commented 3 years ago

Yes, I fixed the errors and made a python script to check for repetitions. I will make a PR.

danyaljj commented 3 years ago

Thank you all!