Make Ores threshold dynamic based current precision/recall - Githubissues

wikimedia / mediawiki-gadgets-RTRC

Mirror of https://gerrit.wikimedia.org/g/mediawiki/gadgets/RTRC.

https://meta.wikimedia.org/wiki/RTRC

MIT License

26 stars 11 forks source link

Make Ores threshold dynamic based current precision/recall #83

Closed Krinkle closed 7 months ago

Krinkle commented 5 years ago

Right now the cut-off for RTRC's display of Ores information (for the damaging model, and reverted model if damaging is unavailable) is at damaging.true >= 0.45.

This should instead be based on the current precision/recall as known to the Ores API.

Relevant links:

API endpoint: https://ores.wikimedia.org/v3/scores/enwiki?models=damaging&model_info=statistics.thresholds.true
Visual interface to the API endpoint: https://wiki-ai.github.io/ores-reference-ui/dist/
Explanation of precision and recall: https://en.wikipedia.org/wiki/Precision_and_recall
Example of how to use the data from this endpoint: https://github.com/wikimedia/mediawiki-extensions-ORES/blob/a6d853eca91fb5c72b894509bac7644cc7d3a290/extension.json#L209-L217
Example of what the outcome: https://en.wikipedia.org/wiki/Special:ORESModels

/cc @halfak @adamwight

Krinkle commented 7 months ago

Exported to https://phabricator.wikimedia.org/T361290.