Scoring of empty anchoring

[namednil]

It makes a difference for scoring if I use a node like this {"id":3,"label":"break-01","anchors":[]} or like this {"id":3,"label":"break-01"}. Is that intended? Even for smatch?

[oepen]

that contrast does not look like a meaningful difference to me, no! so, the "anchors" counts in the MRP output differ, as do the total counts from SMATCH?

this is the unloved area of technically distinct representations that arguably should be considered equivalent: no ‘anchors’ value, null, or an empty list.

i imagine we should address this in normalization prior to scoring, and possibly also issue a warning in the validator. in my view, no value (i.e. complete absence of the property) is the canonical encoding, when there actually is no information.

[namednil]

Yes. Here's an example:

g1.mrp: {"edges":[{"label":"ARG1","source":1,"target":2},{"label":"op2","source":0,"target":3},{"label":"ARG1","source":3,"target":4},{"label":"op1","source":0,"target":1}],"flavor":2,"framework":"amr","id":"bolt-eng-DF-170-181103-8882762_0111.33","input":"Lowering wages/Breaking Unions.","nodes":[{"id":0,"label":"slash"},{"id":1,"label":"lower-05"},{"id":2,"label":"wage"},{"id":3,"label":"break-01"},{"id":4,"label":"union"}],"time":"2019-04-10 (20:10)","tops":[0],"version":"0.9"} g2.mrp: {"edges":[{"label":"ARG1","source":1,"target":2},{"label":"op2","source":0,"target":3},{"label":"ARG1","source":3,"target":4},{"label":"op1","source":0,"target":1}],"flavor":2,"framework":"amr","id":"bolt-eng-DF-170-181103-8882762_0111.33","input":"Lowering wages/Breaking Unions.","nodes":[{"id":0,"label":"slash"},{"id":1,"label":"lower-05"},{"id":2,"label":"wage"},{"id":3,"label":"break-01", "anchors" : []},{"id":4,"label":"union"}],"time":"2019-04-10 (20:10)","tops":[0],"version":"0.9"} and the outputs of smatch scoring, treating g1.mrp as gold graphs:

{"n": 1,
 "g": 10,
 "s": 11,
 "c": 10,
 "p": 0.9090909090909091,
 "r": 1.0,
 "f": 0.9523809523809523,
 "time": 0.0016243420000000564}

and MRP scoring:

{"n": 1,
 "exact": 1,
 "tops": {"g": 1, "s": 1, "c": 1, "p": 1.0, "r": 1.0, "f": 1.0},
 "labels": {"g": 5, "s": 5, "c": 5, "p": 1.0, "r": 1.0, "f": 1.0},
 "properties": {"g": 0, "s": 0, "c": 0, "p": 0.0, "r": 0.0, "f": 0.0},
 "anchors": {"g": 0, "s": 1, "c": 0, "p": 0.0, "r": 0.0, "f": 0.0},
 "edges": {"g": 4, "s": 4, "c": 4, "p": 1.0, "r": 1.0, "f": 1.0},
 "attributes": {"g": 0, "s": 0, "c": 0, "p": 0.0, "r": 0.0, "f": 0.0},
 "all": {"g": 10, "s": 11, "c": 10, "p": 0.9090909090909091, "r": 1.0, "f": 0.9523809523809523},
 "time": 0.004917088000000014}