Count disagree and pass votes differently

[saffronh]

Problem: Disagree and pass votes are counted the same in the group informed consensus metric (i.e. only % agrees matter, iirc). They shouldn't necessarily be weighted the same way, though.

Suggested solution: Create a way to downweight the pass votes compared to the disagree votes such that they are directionally the same impact on group informed consensus but not the same amount (since perhaps a pass should count for e.g. 1/2 of a disagree).

[metasoarous]

Hi @saffronh. Thanks for submitting this idea.

I think this would be best implemented in custom analyses for those interested in looking at it. I worry it would create confusion for users trying to sort out which "version" of the metric they were looking at while interpreting results if this was togglable, whereas custom analyses can just be explicit that it's using a modified version of the metric.

Taking a step back, I think there's much philosophical debate to be had generally about how to handle pass votes in different situations. I do see the argument that between two comments which a given group agreed on at the same rate, we'd rather highlight the comment which had a fewer disagrees (as apposed to passes). But I wonder here if it doesn't make more sense to tinker with an alternative metric that explicitly penalizes disagreement? I'll think about this a bit more.

Thanks again