Closed oepen closed 5 years ago
I tried this approach, but just using the label as a proxy for edge identities does not seem to be enough to guarantee a valid correspondence – the is_valid
assertion fails for graph #20003013 from data/sample/ucca/wsj.mrp
. Any other ideas?
did you use the triple of (src, tgt, lab) as an identity proxy, or just the label (as your comment seems to suggest)?
Sorry, I misunderstood your proposed solution ... Could you have a look at mces.py
in the branch issue13
and let me know whether this is what you had in mind? (I have not fixed Graph.score()
at this point.)
ah, looking at your commit, i now feel i may have misled you. UCCA edge labels are (very much) not functional, so your proxy is probably too ambiguous, and my initial suggestion of using source–target–label triples to identify edges is not compatible with the framework of pseudo-edges. so these need to be real edges, probably parallel to the labeled edge itself, but using a tripartite ‘label’: label–attribute–value?
Yes, I think I arrived at a similar conclusion. At this point however, the MCES scorer does not support labeled edges in the input graph, but my proposed implementation (in the branch) uses them for edge attributes in the InternalGraph
.
i looked at your branch, and was going to ask about edge labels: i thought we were enforcing matching labels for edge–edge correspondences in MCES, but that does not currently seem to be the case?
either way, i think the edge itself should count for one point, and so should each edge attribute. in other words, i suspect the current ‘else’ clause should always be run?
Correct, we are currently considering only the unlabeled case. With the branch, it is easy to support the labeled case however. If you agree then I will merge the branch with master
and make the labeled case the default.
yes, please go ahead! i expect this should make MCES more efficient. we do say on the task web site (on the evaluation page):
However, types (E) and (F) will not be considered separately (i.e. edges and their labels), in part because it does not appear desirable to try and give credit for edges with incompatible lables (e.g. an ARG1 with an ARG3), in part because it makes the search for node-to-node correspondences somewhat more tractable.
i still need to make Graph.score() consider edge attributes.
the current MCES implementation does not yet consider edge attributes (present only in UCCA, to distinguish remote from primary edges) in its search for the best node-to-node correspondence relation. i imagine it might just work to add pseudo-edges much like for node properties and such, maybe using a triple comprised of the edge source, target, and label as a proxy for edge identities? the Graph.score() will also need to be extended, to count and return an additional type of tuples, say called ‘attributes’.