search

DLu / ros_metrics

14 stars 6 forks source link

Show 'helpful' ratio on Answers section #11

Open gavanderhoorn opened 4 years ago

gavanderhoorn commented 4 years ago

metrorobots.com/answers/users had this.

Would be nice for metrics.ros.org to also show it.

mintar commented 4 years ago

Really? I rather find it a useless score (at least the way it's currently defined), since it gives everyone with at least 1 answer but 0 questions a helpful ratio of ∞, whereas people like Tully and Austin with hundreds of answers and a handful of questions start around rank 680.

gavanderhoorn commented 4 years ago

I'm not saying we should make a 1-to-1 copy. Obviously filtering out some 'nonsense' results would make sense.

DLu commented 4 years ago

I'm open to hearing different formulas.

mintar commented 4 years ago

This would already be an improvement:

n_answers / (n_questions + 1)

Asymptotically, it's a good approximation for n_answers / n_questions, while avoiding the "divide by zero" problem. This still has the problem that it doesn't account for the total number of questions/answers by a user, so a user with 10 answers and 5 questions will be ranked higher than a user with 99 answers and 60 questions.

Since we strangely seem to want a metric that rewards answers while punishing questions, we could treat answers as "upvotes" and questions as "downvotes" and use reddit's metric (the lower bound of Wilson score confidence interval for a Bernoulli parameter). There's even an xkcd blog post about this.

Oh, and since you said you want formulas, here's the formula:

equation

Or, you know, we could just stick with the karma points of the user, which doesn't punish questions but instead even rewards good questions! :)