We should have a page that allows us to see worker statistics and other information. For example:
Number of answers by a judge on a particular evaluation, and overall
How often the judge differs from the majority response (for questions where multiple judges are allowed to answer). If we want to get fancy, we could add an EM algorithm, but I think this is overkill because most judges are trusted.
Judge speed. (I don't think this is that useful, because some judges simply work much faster, and Amazon doesn't seem to always correctly measure time. But sometimes it's helpful.) How much total money we've paid them (just an interesting fact).
Add comments about workers (e.g., I interact a lot with our workers, so I know extra information about them). Give them a friendly name.
We should have a page that allows us to see worker statistics and other information. For example: