search

neotenic / protobowl

real time multiplayer quizbowl
http://protobowl.com/hsquizbowl
Other
95 stars 27 forks source link

Checking that works more than three fourths of the time #76

Open cchan opened 10 years ago

cchan commented 10 years ago

For example, when given a multi-select whose answer is "4 2 5 3 1" it accepts any permutation which is pointless. And random stuff happens too - carboxyl is very different from carbonyl, but they're only one letter apart.

[what happens to all of those "report answer as wrong"-s anyway?]

cchan commented 9 years ago

By the way, do you have a dataset of "correct" answers versus user answers? I would love to run a bit of machine learning on it if so, and see if I can get some good accuracy. Thanks.

alchzh commented 9 years ago

Hmm, that might be a bit complicated, and for questions that are very often mistaken that could be a problem. It's a cool idea, but I'm not sure if it is possible.

NielsKornerup commented 9 years ago

What might work (I don't know if this is implemented) is a system where protested incorrect answers are compared and duplicates are placed somewhere where a human can evaluate whether or not to accept them.

cchan commented 9 years ago

I think it exists. The protests certainly seem to be submitted when you click on the little "Correct" and "Incorrect" labels, but I have no idea where those actually go. There's also those Edit buttons on the questions themselves. (@polarcuke @antimatter15 ?)

So that protest system with human evaluation would be indeed necessary, but if you add a machine learning system to it (plain supervised learning, presumably) it could gradually get better and better. Or possibly not, since the features are pretty complex. Lol, I'll try.