search

rifflearning / zenhub

This is the master repository for the Riff Projects in our ZenHub Workspace
0 stars 0 forks source link

Collective Intelligence Metric #51

Open adonahue opened 5 years ago

adonahue commented 5 years ago

As PM, I would like to add a Collective Intelligence metric to the Riff dashboard, so that we provide additional value about the nature of peoples' conversations.

Research summary and implementation details provided by @jaedoucette, here: https://docs.google.com/document/d/100ALiqYpZ10ON7lkaBdh8a2epcC35jxIlOc7UjZuiLU/edit?ts=5cf924ca#heading=h.tli2vt7k678v

Conceptual Graphic image.png

brecriffs commented 5 years ago

Very cool! I don't completely understand the Math, but luckily that isn't my job! It might be cool to show the individual intelligence with (u/T-N)*(u/T-N)?

juliariffgit commented 5 years ago

This is so interesting! Is this just turn-taking or does influencing also come into play?

jaedoucette commented 5 years ago

Update: my math was wrong for that scaling-factor part at the end. I think that it'll be under 0.25 in practice, but we might get some really weird cases where it's larger than that. The fix is in two parts. First, use maxV = 0.5. Second, if an individual user's contribution to V is larger than 0.5, just cap it at 0.5.

That kinda hacky. Basically it'll say that any conversation where one person does like 90%+ of the talking gets a score of 0, when really we ought to give it a score of like 0.5 or 1 or something. It should be close enough to give the user good information though.

jaedoucette commented 5 years ago

Regarding the individual scores, yeah, I thought about that. I think the ideal way to visualize it is to graph $DesiredScoreRange/2*max(0.5,(u/T-N))/sqrt(maxV)$ for each user.

Scores from -5 to 0 mean you're not talking enough. Scores from 0 to 5 mean you're talking too much.

The individual scores don't really have a meaning in terms of individual intelligence, but they could help people visualize how their behavior might be affecting the intelligence of the group. We'd want to be pretty careful about communicating that distinction.

brecriffs commented 5 years ago

Hmm yeah! Would be cool to maybe compare how your individual score affects group scores historically. Like "Hey, when you talk too much it negatively affects your group's score". But there could be some cases where a person talking a little bit more than the average helps the group score. Something like a scrum master ;) That would be helpful to know as well!