RED+AMR annotation discussion on whether to annotate number

[timjogorman]

The Argument for encoding number

Sorry for the deluge of posts here. Here's a set of arguments for why we want to be annotating some kind of number or quantification in the multi-sentence AMR pass.

General claim:

• Many might claim that our conventionalizing of singular/plural distinctions, of specific vs generic, is an issue
  • Most languages do do some kind of coding for plurality. WALS has a good summary (the Chinese dialects are in the <10% that don't):
  • 
• That doesn't necessary mean that AMRs is wrong in throwing out English grammatical plurality
  • We don't want to encode language-specific categories like mass/count
  • even with logic-form, we'd run into variance ( "every farmer who has a donkey beats it" vs "farmers with donkeys beat them") and cumulative/"dependent" plurals ("Unicycles have wheels")
  • often ambiguous within a single-sentence context
• General question: can we have a simple encoding on entity chains that gives some deeper, KB-level understanding of number?
  • Vindicate our sentence-level decisions with something more objective
  • possibly: Make actual logic-form approximation viable
• Proposed solution:
  • Let's just think about number in a more "KB"-centric way
  • "If you were to quantify this discourse referent, at a document level, is there only one of it?"
  • Remember: currently we know very little about our variables. "m / man" means "one or any other number of men, who may or may not exist". Can we cheaply start solving that during annotation?

    Implementation proposal

-"Entity Coreference" as a single kind of chain. This is the same thing we did in the last pass, but you have three different kinds of "mentions".

• Unique: If (m / man) is in that box, the discourse referent is a unique entity of type "man"
• Multiple: If (m / man) is in that box, the discourse referent is a set, and the members are of type "man"
• FullDefinition: Everything described by this mention is part of the discourse referent. If "(m / man) is in that box, it refers to all men. .

  What would "Unique" mean?

  • That for "c / concept", c refers to a single "thing" -- there aren't many "c"s
  • "Every boy has a favorite toy. It's the spacecat 9000". The "a favorite toy" is an actual, single referent in the world (the "a favorite toy" has maximually wide scope).
  • This means that we don't do "unique" for "Every boy has a different favorite toy" -- 't / toy' refers to millions of different referents at once.
  • (Another thing this gets us: every "unique" thing identified, you know all of its scope relations to everything else in the document. This makes discussion of annotating scope/quantification a viable goal)

    What would "Multiple" mean?

  • For "c / concept", there are likely more than one different individuals "c", each of type "concept" (but "c" doesn't refer to every single thing of type "concept")
  • We would want to do this for grammatically singular things that are quantified over
  • "Every kid in the class got [a kitten]". (cf "all the kids in the class got kittens")
  • We'd want to do it for everything that is quantified or aggregate, i.e. masses.
  • "Every kid in the class got [candy]". (cf "every kid in the class got candies")

    What would "FullDefinition" mean?

  • For "c / concept", "c" refers to (more or less) every single thing that matches the definition in "c".
  • "[Every kid in the class] got a kitten".
  • If you are making reference chains, knowing which one is the definitional one useful for more than just logic form
  • Cleaner than linguistic ideas like "first mention".
  • Could use it sparingly (mostly for quantification)
  • Generics are hard: Recent work in "generics" (Friedrich et al 2015) make the smart decision of splitting up "reference to kinds" from whether you're talking about "characterizing" statements. This means that "generics" like "John drives to work" can be differentiated from generics like "Lions eat meat".

    Revisiting the coreference chain we've discussed.

  • Remember we had lines like:
  • "White man subjected to race attack by gang of yobs"
  • "One of his attackers then stood over him and..."
  • "...called off when one of the thugs said they should leave..."
  • Unique: "g / gang"
  • Multiple: "t / thug", "y / yob", "p / person (:arg0-of attack-01)", "t / they" ...
  • Assumed 'output' (information we're storing):

 X / entity
           :instance-of "gang" 
           :constist-of (X2 / entity 
                     :instance-of thug
                     :instance-of person
                     :instance-of yob
                     :quant 4

[nschneid]

I think this is an important discussion to have; thanks for getting it started!

One case that comes to mind is a sentence in a paper draft I saw recently: "This table shows the performances of the systems." I understand why "performances" was pluralized here (each system has a different score), but "performance"—aggregating over all systems—sounds more natural to me (I guess I prefer it to be a mass noun). Would annotators have to decide whether "performance(s)" is unique or multiple? Or would that not count as a discourse referent?

[nschneid]

Another question: As I understand it, Unique as described above would include collective nouns (like "gang" in "gang of thugs") as well as non-collective individual referents (like "president"). Should these be separated out? It might be more natural to think of two dimensions:

	unique	multiple
individual	president	gang(s) of thugs
collective	gang of thugs	gangs of thugs

[timjogorman]

"This table shows the performances of the systems."

I'd tend to want "performance" to be an event, which will require different distinctions. I'll try to get that Events pass up this week.

Should these be separated out? It might be more natural to think of two dimensions:

My own instinct is that having them together gives us a slightly more robust approach (we don't need to classify things into "collective nouns" or anything like that), but it might be worth discussing on the call

[nschneid]

My own instinct is that having them together gives us a slightly more robust approach (we don't need to classify things into "collective nouns" or anything like that), but it might be worth discussing on the call

[Sorry I couldn't join the call (was on an airplane).]

What would be the treatment of "gangs of thugs" if only the unique/multiple distinction is available? Wouldn't 'gangs' and 'thugs' both be multiple? Or, more precisely: multiple gangs, each consisting of multiple thugs?