own-pt / rte-sick

RTE Experiment
1 stars 3 forks source link

Appositives in SICK #12

Closed vcvpaiva closed 7 years ago

vcvpaiva commented 7 years ago

Stats say (line 120) 18 appositives, but the paper said there shouldn't be any.

Are the 18 sentences wrong? @livyreal help here?

the appositives in the Turku interface are:

  1. a man performs a large jump . (no appos, perform considered a noun)
  2. a yellow dog anticipates a ball (no appos, antecipates considered a noun)
  3. two women , one in white , one in black , are sitting on a bench. (appos, it seems to me)
  4. There is no cat sliding upside down under a couch (no appos, sliding considered a noun)
  5. Group of schoolchildren in uniforms standing at a gate , one kissing her mother goodbye (no appos, conj?)
  6. A group of children playing in a yard , a man in the background (no appos, conj)
  7. two young women , one with folded arms , look off screen (appos)
  8. Four middle eastern children , three girls and one boy , are climbing on the grotto with a pink interior (appos)
  9. blond child with dirty face holding yellow bottle with red cap , plants in background (no appos, conj?)
  10. a man with a white hat and brown shirt , behind a woman in red , looks to the left (appos)
  11. Two large dogs , a light colored one and a dark one , are quickly moving through snowy terrain (appos)
  12. A cowgirl corners a barrel riding a horse (no appos? conj?)
  13. Two large dogs , a light colored one and a dark one , are running through snowy terrain (appos)
  14. a man in seattle holds a sign begging for money (no appos, holds considered a noun)
  15. two men , one in a blue swimcap , stand in torso-deep water (appos)
  16. two adults - a male and a female - are standing on scooters in a field of yellow flowers (appos)
  17. a cat , perched on a table , looks out the window (appos)
  18. A man in a shirt dyed purple is walking away from a man in a black shirt , who is serious (??)
  19. three small dogs sniff at somthing (not appos, sniff is adj)
  20. Two large dogs , a light colored one and a dark one , are sleeping on a snowy terrain
  21. a little boy wearing a yellow tank top is laughing (???)
  22. a little girl swings , a woman stands behind her (no appos, conj)
  23. a man rode a mechanical bull (no appos, rode considered noun
  24. A young woman , topless except for paint or mud (no appos?)
  25. topless woman smeared with brown substance , blurry crowd in background (no appos)
  26. three men sitting on a hill looking at the mountains and sky (no appos, conj?)

anyways quite a few (8) more appositives in the Turku interface than in the stats, why?

fcbr commented 7 years ago

The Turku interface uses @arademaker's deduplicated sentences (which also include the original SICK phrases). Initially I only processed the derivative sentences. Anyway, I have updated the stats.xml with the results from conll/sentences.conllu.

I'm not sure if we should have included the original SICK sentences since they are not used in the entailment analysis, but maybe this is subject for a different issue.

vcvpaiva commented 7 years ago

Anyway, I have updated the stats.xml with the results from conll/sentences.conllu.

thanks!

and yes maybe the original sentences should've been kept apart: in one part of the experiment we focus on TE and on the other on conceptual semantics.

vcvpaiva commented 7 years ago

on the other hand there are appositives in the corpus that were not caught, e.g. https://github.com/own-pt/rte-sick/blob/master/pairs/1032.txt A person who is practicing snowboarding jumps into the air. agreed @livyreal ?

vcvpaiva commented 7 years ago

Appositives in SICK-PARSEY seem better. stats say 23. https://github.com/own-pt/rte-sick/blob/master/conll/sentences-parsey.conllu.stats.xml Turku interface provides them all, mostly correct.

some are clearly correct appositives like two young women , one with folded arms , look off screen four middle eastern children , three girls and one boy , climb on a grotto with a pink interior

some are debatable such as A group of children playing in a yard , a man in the background The back of a woman in costume ; little girl looks at her I would've thought conjunction both and some others.

One or two real errors: a small child climbs steps outdoors in a grassy area (no appos, climbs=verb, steps=noun, not vice-versa) blond child with dirty face holding yellow bottle with red cap , plants in background (root shouldn't be child, but holding.)

arademaker commented 7 years ago

We have now '[A person] [who is practicing snowboarding] jumps into the air.' with rcmod(practicing, person). Not sure if rcmod is the UD appos.

fcbr commented 7 years ago

We need to check the origin of the dependency tag set of Parsey McParseface. If it is the same as the original Stanford dependencies, then rcmod is:

rcmod: relative clause modifier

A relative clause modifier of an NP is a relative clause modifying the NP. 
The relation points from the head noun of the NP to the head of the
relative clause, normally a verb.

“I saw the man you love” rcmod(man, love)
“I saw the book which you bought” rcmod(book,bought)
fcbr commented 7 years ago

According to their paper, section 4.2 Dependency Parsing: Our model configuration is basically the same as the one originally proposed by Chen and Manning (2014) and then refined by Weiss et al. (2015).

vcvpaiva commented 7 years ago

@arademaker said:

Not sure if rcmod is the UD appos.

but we do have appositives in SICK-PARSEY, I don't get it.

EDIT: I got it now, @arademaker talked about the example that I was asking @livyreal if it was an appositive, for her. @fcbr showed it it's not, it's considered relative clause by SD.

vcvpaiva commented 7 years ago

if the creators of the corpus had succeed completely there should be no appositives in this corpus.

listing the ones marked by parsey in the interface, query <appos that seem to me correct:

  1. two women , one in white , one in black , are sitting on a bench
  2. two large dogs , one light colored and one dark , running through snowy terrain (NOT a sentence!, remove? #19 )
  3. Two large dogs , a light colored one and a dark one , are quickly moving through snowy terrain
  4. Two large dogs , a light colored one and a dark one , are running through snowy terrain
  5. Two large dogs , a light colored one and a dark one , are sleeping on a snowy terrain
  6. two young women , one with folded arms , look off screen
  7. Four middle eastern children , three girls and one boy , are climbing on the grotto with a pink interior
  8. four middle eastern children , three girls and one boy , climb on a grotto with a pink interior
  9. two men , one in a blue swimcap , stand in torso-deep water

The ones that don't look like appositives to me:

  1. A young girl is wearing a blue patterned swim suit , a black and yellow swim cap and has blue swimming goggles on her head. (appos(suit, cap)??)
  2. A little girl is wearing a blue patterned swim suit , a black and yellow swim cap and has blue swimming goggles on her head
  3. a young girl is wearing a blue patterned swim suit , a black and yellow swim cap and has blue swim goggles on her head
  4. A young boy is wearing a blue patterned swim suit , a black and yellow swim cap and has blue swim goggles on
  5. A group of children playing in a yard , a man in the background (appos(group, man)??)
  6. blond child with dirty face holding yellow bottle with red cap , plants in background (appos(bottle,red cap)??)
  7. three hikers are sitting on a mountainside looking at the view , one using a pair of binoculars (appos(mountainside,one)??)
  8. in rural outdoors , blond woman sits on roof of yellow bmw vehicle , two people inside
  9. there are two guys sitting around a table , both with beers in their hands
  10. The back of a woman in costume ; little girl looks at her
  11. distant person with blue backpack hiking in rocky area , mountains in background
  12. topless woman smeared with brown substance , blurry crowd in background
  13. a small child climbs steps outdoors in a grassy area
vcvpaiva commented 7 years ago

anyways 23 in 7.9K is a small number, closing this issue, hoping that this is small enough.