SputLink updates

[marcverhagen]

SputLink is integrated and creates new links, but there are issues:

• [x] We now get link duplication because existing links are added again. Need to remove all existing links from the TarsqiDocument before we add the new batch, this is also needed because some of the original links may have been deemed to be inconsistent.
• [ ] Decide what we want to take from the output. We now take all non-disjunctive links. We could add the disjunctive links as well. Or we could not take inverse relations. or we could reduce the graph to a minimal graph.
• [x] We now put all links on the queue, but they are not ordered. For the merging routine to be fully effective we should rank the TLINKs in terms of how likely they are to be correct.
• [ ] Since commit 102275a4fb15193a24691fbdfff4d5524bcb04fb we now put all links on the queue, and order them in a rather simplistic way (S2T < Blinker < Classifier, and for classifier links we also use the confidence score. For the merging routine to be better we should let the ranking be informed by hard evaluation data.
• [x] Write logger warnings when a link was not added due to inconsistencies.
• [ ] From the queue, we add a link, run closure, then add the inverse of the link and run closure again. It may be possible to get a good speed up by adding the link and run closure, but keep track of everything added and then add the inverses as well.
• [ ] Think about how to implement reference intervals

[reevesr]

Doesn't the TLINK ranking depend on probabilities generated by Classifier?

[marcverhagen]

In theory, yes, in practice, not in this version of the code. But it is easy to add. I am actually writing an email right now to you and Josh on this issue and other issues surrounding SputLink.