search

oughtinc / mosaic

A web app for recursive question-answering with pointers
MIT License
37 stars 3 forks source link

Have the ability to assign whether someone is a judge or not #518

Open brachbach opened 5 years ago

brachbach commented 5 years ago

either per experiment or per tree, no strong view on which is better

would be very slightly useful now for experimenter runner to be able to assign themselves as expert but not judge in an experiment

I anticipate will be more useful in the future when I expect contractors to specialize into the different roles

stuhlmueller commented 5 years ago

Also want notifications for judges eventually (e.g. randomly selecting a judge to notify, and sending more notifications to other judges if the first one doesn't respond).

brachbach commented 5 years ago

Value: 2 for now, may be higher value in the future once we want contractors to specialize in specific roles

Mati-Roy commented 5 years ago

This is done, right? Given non-HE non-ME are reputated having been assigned as J. Or is it that we want the ability to also assigned someone to no role?

zjmiller commented 5 years ago

Yeah the idea is to switch from having the judge role be the default to having the judge role be something that you have to actively give to someone.

Mati-Roy commented 5 years ago

another advantage of this is that we could assign participants to only, say, 75% of trees in order to allow them to analyze 25% of them without being info contaminated

alternatively, but mostly equivalently, we could add the role of analyst