As an additional incentive to turnaround reviews faster, we should be able to query specific reviewers' statistics.
We should save:
Total reviews completed
Total of review results (approved, rejected, commented, pending)
Average response time (save in milliseconds but render as days)
Also as a line of defense for their reputation I have a couple of ideas:
If they dismiss themselves then the review statistic is not recorded.
Grace Period: perhaps it makes sense to provide a 24 hour grace period (configurable on the repository level!) where if a review is requested from a reviewer, they have 24 hours to dismiss themselves (basically refusing to do the review.)
The timer should only start if they are requested for a review, and more than 24 hours has passed to come to a conclusion (merged or closed.)
This should be compatible with the bot automatically closing pull requests for low performing bounty hunters.
If the review is merged within the grace period (24 hours) perhaps we should not update the reviewer statistics because it seems like a bogus review.
Combination of both ideas: review statistics recording is only valid after the grace period. The reviewer can back out at any time without recording the review statistics. But if they try to jump in again then the counter needs to start from the first time they were requested for a review, which is probably not great for their stats.
Scopes
It would be very interesting to have this viewable on the repository level, organization level, and global level.
I think repository level is the most useful and relevant for most cases, and organization level as a reference point if they are jumping into a new project within an organization.
I don't think we should make "global" public just yet, but it could be very interesting to have a "god mode" and poach the best reviewers.
Schema
We can handle this with simple SQL queries as long as we store the pull request ID, repository ID, organization ID.
We can calculate averages per scope by filtering out based on the repository and organization IDs.
Review Turnaround Time
As an additional incentive to turnaround reviews faster, we should be able to query specific reviewers' statistics.
We should save:
Also as a line of defense for their reputation I have a couple of ideas:
Scopes
Schema