Task 6 -- Family relations - Memory task: retrieve the relations already defined in the text

[pminervini]

Hi, I think there is something wrong with generating datasets for "Task 6 -- Family relations - Memory task: retrieve the relations already defined in the text" -- If I do e.g.:

$ PYTHONPATH=. python3 main.py --train_tasks 6.2,6.3 --test_tasks 6.2,6.3

I get instances such as this one:

$ tail -n 1 ~/workspace/clutrr/data/data_08aa323e/6.2_test.csv
84,f40ac862-0ba6-4f70-8c3a-9e060edd8bed,[Calvin] is [Henry]'s grandfather.  [Travis] has a son called [Henry]. ,"('Travis', 'Calvin')",Who is [Calvin] from the point of relation of [Travis] ? ,father,['[Calvin] is the father of [Travis]. '],[Travis] has a son called [Henry].  [Calvin] is [Henry]'s grandfather. ,"[{('Travis', 'father', 'Calvin'): [('Travis', 'son', 'Henry'), ('Henry', 'grandfather', 'Calvin')]}]",son-grandfather,task_6.2,"[(0, 1), (1, 2)]","['son', 'grandfather']","(0, 2)","Travis:male,Henry:male,Calvin:male",,"{2: 0, 10: 1, 0: 2}",test

which does not reduce in retrieving relations already defined in the text, but requires some sort of reasoning.

Am I doing anything wrong here?

[koustuvsinha]

Hi @pminervini, sorry for the delayed response, my github notifications hasn't been properly configured it seems. Thanks for raising this issue. I haven't done many experiments with this specific setup so I haven't looked into the details much till now. I'll have a look in this.