linas
commented
5 years ago The following sentences appear two times, or more, in that file:
alice mommy knocked them down
a@l is for apple
all fall down
and what are these
and what are those
and what 's that
and what 's this
and who are you
and who 's that
brush mommy 's hair
can you brush his hair
can you count them
can you do it
can you do that
can you do this one
can you feed it to the doggie
can you get him
can you get him out
can you get it
can you give me a little kiss
can you make something else
can you open it
can you open the door
can you pat the bunny
can you say bye bye
can you smell the flowers
can you stack them up
can you tell me
can you zip it up
daddy has a scratchy face
daddy 's not here
daddy 's scratchy face
do you know what that is
do you know what this is
do you see something you want
do you want this
here are paul and judy
here he is
here it is
here 's another book
here 's one
here we go
here you are
here you go
he 's a mouse
he 's gone
he 's gonna get you
he 's playing peekaboo
he 's sleeping
how are you
how are you today
how does that feel
how many are there
how many do you have
how many is that
is it a dog
is it a kitty
is it daddy 's shoe
is it for me
is that a baby
is that a pig
is that his mouth
is that the dog
it doesn't come off
it 's a book
it 's a bunny
it 's a cat
it 's a cow
it 's a dog
it 's a doggie
it 's a dragon
it 's a duck
it 's a kitty
it 's all gone
it 's a mouse
it 's an apple
it 's another one
it 's a truck
it 's for cindy
it 's good
it 's not a toy
it 's okay
judy can feel daddy 's scratchy face
judy can look in the mirror
judy can pat the bunny
judy can read her book
know what that is
let me out of here
let me out of my cage
let's look at this book
let's sit up here
look at all these fun things
look at the book
look at this book
look in the mirror
looks like a nice one
no it goes this way
now he 's gone
now you pat the bunny
please let me out
put it in his mouth
put it on top
put your finger in the hole
put your finger in the ring
put your hand in here
put your hand in there
say hi to daddy
stick your hand in
take off my shoes
that 's a balloon
that 's a bird
that 's a boy
that 's a brush
that 's a cat
that 's a dog
that 's a duck
that 's a girl
that 's a good book
that 's a good girl
that 's a hole
that 's a kitty
that 's all
that 's all of them
that 's a mouse
that 's an egg
that 's a spoon
that 's a turtle
that 's a zipper
that 's daddy 's scratchy face
that 's good
that 's his hair
that 's mommy 's ring
that 's nice
that 's not a toy
that 's okay
that 's right
that 's the dog 's house
that 's very good
the boy 's turn
there he goes
there he is
there it is
there 's alice
there she is
there 's one
there 's the baby
there 's the boy
there 's the dog
there 's the rabbit
there they are
there we go
there you go
these are daddy 's
they all fell down
they can do lots of things
they 're saying bye bye
those are ducks
those are flowers
wanna put them back
want me to get it
what a fun book
what are these
what are they doing
what are those
what are you doing
what are you doing today
what are you gonna do
what are you looking for
what color is that
what color 's that
what color 's that egg
what color 's that one
what did you do
what did you do today
what does he say
what does that feel like
what does the doggie say
what does this say
what do they say
what do you do with the brush
what do you say
what do you see
what do you see in the mirror
what do you think
what do you think this is
what do you wanna do
what do you want
what else does he have
what is it
what is judy doing
what is that
what is this
what 's bunny doing
what 's here
what 's her name
what 's his name
what 's in here
what 's in his hand
what 's in the house
what 's in there
what 's my name
what 's over there
what 's paul doing
what 's that
what 's that alice
what 's that boy doing
what 's that called
what 's that color
what 's that gail
what 's that lena
what 's that letter
what 's that one
what 's the bunny doing
what 's the cat doing
what 's the doggie gonna say
what 's the doggie 's name
what 's there
what 's this
what 's this book
what 's this called
what 's this gail
what 's this lena
what 's this letter
what 's your name
what was that
what would you like
what would you like now
where are his eyes
where are the flowers
where are you going
where did he go
where did it go
where did she go
where is he
where is she
where 's his mouth
where 's paul
where 's the baby
where 's the boy
where 's the butterfly
where 's the dog
where 's the doggie
where 's the dog 's face
where 's the duck
where 's the girl
where 's your ring
where 's your tongue
which one do you want now
who are you gonna call
who are you talking to
who is that
who 's in that
who 's in that mirror
who 's in the house
who 's in the mirror
who 's in there
who 's on the phone
who 's on the telephone
who 's that
who 's there
who 's this
will you read me a book
would you like that
yeah he 's gone
yeah put it on top
yeah there it is
you can do it
you can do lots of things too
you closed the door
you did it
you did them all
you do it
you don't like that
you don't want to
you knocked it over
you knocked them over
you know what that is
you like his tongue
you like that
you like the blocks
you like those
you look at it
you need some help
you put your finger in there
you put your hand in here
you 're done with that
you 're gonna close the door
you 're gonna put them back
your name is kay
you want him to bite
you want me to do it
you want the ball
you want this
akolonin
This file: https://github.com/singnet/language-learning/blob/master/data/CDS/LG-E-clean/CDS-LG-English-clean.ull
contains several sentences with a non-sense word:
Note that the last sentence is repeated.
Also -- naively, it appears to contain 3037 sentences, but of these, only 1836 are unique; the rest are duplicates.
I have not explored why there are duplicates. Of course, if the duplicates are recording two different parses of the same sentence, then the use of this as a gold standard means that, no matter what, a disagreement will be found for at least one of the duplicates.