Allow recommenders to return probability scores

[pseudomonas]

I've adapted the Sklearn script for POS-tagging, and it'd be really useful for it to return a set of label-score tuples for each token. The create_prediction api doesn't seem to support that though?

[reckart]

What does the tuple represent?

[pseudomonas]

each potential label, and the probability that the classifier assigns to it in that space. (could be a dictionary rather than a tuple; sklearn-crfsuite uses dictionaries)

inception's internal classifiers have scores attached which allows functionality like "ignore recommendations below N probability" and "show potential labels in order of likelihood"; I'd like to get that working for external recommenders too.

[reckart]

Ah, sorry. What you do is, that you create a prediction for each of the label/score tuples.

prediction = create_prediction(cas, layer, feature, begin, end, predicted_label)
prediction.set(f'{feature}_score', score)
prediction.set(f'{feature}_score_explanation', 'Some reason for the score (optional)')
cas.add_annotation(prediction)

[pseudomonas]

Thank you!

(also, is there documentation to this effect somewhere, so I can avoid asking unnecessary questions in future? I couldn't really find anything with the googling that I did.)

[reckart]

Eh... not yet :) ... and to be honest, it should probably simply be something that the create_prediction method offers as optional arguments. If it had these arguments, you probably wouldn't have had to ask.

[pseudomonas]

In that case I'll ask my other question in another Issue thread, and hopefully if someone else has the same question they can find that. BTW, would you like my POS-tagging code once I've made the changes above?

[reckart]

Contributions are always welcome :)

[reckart]

If you like to submit a PR to add the optional parameters to create_prediction, that's also welcome ;)

[reckart]

There are optional parameters to create_span_prediction() and create_relation_prediction() now.