Measure the check news vs flagged news metrics

[rogeriochaves]

So, before deploying our machine learning model, we always check the accuracy scores, f1 scores, etc, but, better than checking beforehand if the model improved or not, is having a real metric from production systems of weather it is working better or not.

So an idea is taking the API logs to measure how much news have been checked by robinho vs how much of them is wrong enough to bother users to click the Flag button and flag 'em

This metric would probably go up if we had much false positives or false negatives, and down if we are getting it right

Of course this metric can be infected, like someone very feeling helpful and flagging everything, or if all checkings are so wrong that users just give up or ignore it, but it is something

[rogeriochaves]

This issue was moved to fake-news-detector/fake-news-detector#12