kevinkwl / AoAReader

PyTorch implementation of Attention-over-Attention Neural Networks for Reading Comprehension
MIT License
61 stars 21 forks source link

what's the data of train.txt, dev.txt, test.txt #3

Closed SeekPoint closed 6 years ago

kevinkwl commented 6 years ago

Children's Book Test dataset from Facebook.

SeekPoint commented 6 years ago

lrwxr-xr-x 1 ke.ke staff 44 12 27 11:19 dev.txt -> ../../../Downloads/CBTest/data/cbt_valid.txt lrwxr-xr-x 1 ke.ke staff 43 12 27 11:13 test.txt -> ../../../Downloads/CBTest/data/cbt_test.txt lrwxr-xr-x 1 ke.ke staff 44 12 27 11:13 train.txt -> ../../../Downloads/CBTest/data/cbt_train.txt

however, I got this error while preprocess.py:

........................................................................... /Users/yike.ke/yike_prj/AoAReader/preprocess.py in parse_stories(lines=['_BOOKTITLE : Andrew_Lang___Prince_Prigio.txt.out\n', 'CHAPTER I. -LCB- Chapter heading picture : p1.jpg -RCB- How the Fairies were not Invited to Court .\n', 'Once upon a time there reigned in Pantouflia a king and a queen .\n', 'With almost everything else to make them happy , they wanted one thing : they had no children .\n', 'This vexed the king even more than the queen , w..., and who had hated dolls when she was a child .\n', 'However , she , too in spite of all the books sh...lad enough to be the mother of a little prince .\n', 'The king was anxious to consult the fairies , but the queen would not hear of such a thing .\n', 'She did not believe in fairies : she said that t...Family was full of chapters about nothing else .\n', 'Well , at long and at last they had a little boy...ded as the finest baby that had ever been seen .\n', 'Even her majesty herself remarked that , though ...ertainly was a fine child -- a very fine child .\n', 'Now , the time drew near for the christening par...akfast in their summer parlour talking over it .\n', 'It was a splendid room , hung with portraits of the royal ancestors .\n', 'There was Cinderella , the grandmother of the re...oot in her glass slipper thrust out before her .\n', 'There was the Marquis de Carabas , who , as ever...ge with the daughter of the king of the period .\n', 'On the arm of the throne was seated his celebrated cat , wearing boots .\n', 'There , too , was a portrait of a beautiful lady...rmant , also an ancestress of the royal family .\n', 'Many other pictures of celebrated persons were hanging on the walls .\n', "You have asked all the right people , my dear ? ''\n", 'said the king .\n', " Everyone who should be asked , '' answered the queen .\n", ...], withanswer=True) 37 for line in lines: 38 line = line.strip() 39 if not line: 40 story = [] 41 else: ---> 42 , line = line.split(' ', 1) _ = 'You' line = '.' line.split = 43 if line: 44 if '\t' in line: # query line 45 answer = '' 46 if with_answer:

ValueError: not enough values to unpack (expected 2, got 1)

kevinkwl commented 6 years ago

The three files you are using are the divided stories, use the data files start with 'cbtest_'

SeekPoint commented 6 years ago

it works