search

johnwdubois / rezonator

Rezonator: Dynamics of human engagement
35 stars 2 forks source link

Gold standard #299

Open johnwdubois opened 5 years ago

johnwdubois commented 5 years ago

Is your feature request related to a problem? For evaluating game scores and tutorial quizzes, we need to be able to compare the user's (or player's) input with the "correct" answer. This is the Gold Standard.
Also, we need to provide tools for Content Creators, researchers, and power users that make it easy for them to propose a Gold Standard (the "correct" answer) for all kinds of links and chains in Rezonator.
It should also be easy to verify the Gold Standard by measuring inter-annotator reliability.

Describe the solution you'd like

  1. Create a field in all Link and Chain tables that allows a user to "sign" their work with a UserId. (This is #132 ).
  2. One of these markups can be proposed as a Gold (gold standard).
  3. Create a secure mechanism for verifying a proposed markup as a true Gold (once it passes the test, it is then validated as Gold ).
  4. Lower levels of validation can be designated as silver or bronze.

Additional context Related to #287 #132 #135 #144.

johnwdubois commented 3 years ago

It is useful to restore the gold standard in Rezonator, for both experiments and games.