search

coli-saar / am-parser

Modular implementation of an AM dependency parser in AllenNLP.
Apache License 2.0
30 stars 10 forks source link

Invalid AMR output? #82

Open sctratz opened 4 years ago

sctratz commented 4 years ago

Hello.

I'm producing AMRs using your system and I'm getting at least one that smatch.py doesn't like. Any thoughts?

Input (raw text): if you recall from our previous mission we built an open 3x3 square as the second level of our " table

Output AMR: (u_1701 / build-01 :ARG0 (u_1713 / we) :ARG1 (u_1702 / square :quant (u_1700 / 3x3) :ARG1-of (u_1703 / open-04)) :prep-as (u_1712 / level :poss (u_1707 / table :mod (u_1704 / "\"")) :ord (u_1706 / ordinal-entity :value (u_1711 / name :op1 second))) :condition (u_1705 / recall-02 :ARG0 (u_1708 / you) :time (u_1709 / mission :mod (u_1710 / previous))))

Reference AMR: (b / build-01 :ARG0 (w / we) :ARG1 (s / square :ARG1-of (o / open-06) :mod (d / dimension-entity :value 3) :mod (d2 / dimension-entity :value 3)) :ARG4 (l / level :ord (o2 / ordinal-entity :value 2) :part-of (t / table :poss w)) :ARG2-of (r / recall-02 :ARG0 (y / you) :ARG1 (m / mission-01 :ARG0 w :time (p / previous)) :ARG1-of (p2 / possible-01)))

Smatch.py output: Error: Node name not found u_1704 Traceback (most recent call last): File "/home/stratz/anaconda3/bin/smatch.py", line 889, in main(args) File "/home/stratz/anaconda3/bin/smatch.py", line 846, in main justrelation=arguments.justrelation): File "/home/stratz/anaconda3/bin/smatch.py", line 804, in score_amr_pairs justrelation=justrelation) File "/home/stratz/anaconda3/bin/smatch.py", line 735, in get_amr_match amr2.rename_node(prefix2) AttributeError: 'NoneType' object has no attribute 'rename_node'

namednil commented 4 years ago

Hi Stephen,

it's the node u_1704. Apparently, our implementation of the node labels differs in the edge case that a quotation mark is part of the label. A quick and easy fix might be to replace """ by "[insert some other variant of quotation mark here]" with sed or delete that node with a regular expression. I couldn't find any instance of such a quotation mark as part of a node label in the AMR corpus, so I don't know how it is meant to be represented.

alexanderkoller commented 4 years ago

Let's perhaps document this as a limitation of our system in a suitable place.

@sctratz If this is sufficiently important to you, we would really appreciate it if you could dig up some information on how smatch.py wants quotation marks quoted in its input. Then we could probably add code to quote them correctly on our end.