bellroy / predictionbook

Find out just how sure you should be, and get better at being only as sure as the facts justify.
http://predictionbook.com/
72 stars 11 forks source link

Withdrawn predictions count toward the results on the graph? #110

Open QuadKat opened 7 years ago

QuadKat commented 7 years ago

A withdrawn prediction is included in my score in the accuracy box for the range it's in.

First, I made a prediction and judged it true, then unknown and withdrawn. It was my only prediction at the time. I noticed that the withdrawn prediction showed up on my results graph.

To see if the withdrawn prediction would drag down my accuracy score, I did a test. I added another prediction to the same accuracy range and marked it correct. The withdrawn prediction is dragging the accuracy score down to 50%.

Perhaps I don't understand what "withdrawn" means in the context of PredictionBook. If so, then maybe my confusion a sign of a user interface bug. It seems like withdrawn predictions just aren't supposed to do that, though.

http://predictionbook.com/predictions/180230

Note: The withdrawn prediction always had a 1% probability, and it went into the 90% accuracy box. This was confusing until I realized that one could make the graph smaller by putting all the predictions with probabilities under 50% into the other boxes and then using a reversed interpretation of the judgements for those. I tested this using a prediction with a 30% probability, then marked it false, and it updated the 70% box to add to my accuracy there. It looks like this is the way it's supposed to be, and that seems cool now that I've figured it out. My confusion might be a common experience for new users. It might be worth adding my experience of confusion to a list of things to do UX testing about.

ghost commented 7 years ago

"I made a prediction and judged it true, then unknown and withdrawn."

updated my stats

reproduced this issue However it's a duplicate of #100 - the 'bug' aspect here fails when you switch to unknown, rather than when you withdraw it. (so this particular bug can be closed as a duplicate)