search

mozilla / aestimia

[Archived] Assessment tool
5 stars 10 forks source link

Implement Ability to Revise Reviews #75

Open chloeatplay opened 11 years ago

chloeatplay commented 11 years ago

Problem: "I noticed that a submission I intended to approve was rejected (I must not have checked all the right boxes) and I couldn't go back and fix it. Is it possible to revise reviews?"

toolness commented 11 years ago

The hard part here is the fact that Aestimia immediately calls up CSOL-site once a review is submitted, at which point CSOL-site will check to see whether the assessment was successful or not, and email the applicant accordingly (at least, I think that's what CSOL-site will do). Changing the specific feedback of a review to e.g. include a detail one forgot to mention is one thing, but changing the award/reject outcome is another, because by the time the mentor makes the change, an email about their previous decision will already have been sent.

So, there's a few possible solutions here:

  1. Aestimia doesn't even inform CSOL-site about the outcome of an assessment until some amount of time has passed, e.g. 15 minutes. This gives the assessor a window of opportunity in which they can revise any mistakes they made.
  2. As the original bug reporter mentioned, it was actually the mentor's original intent to approve the submission, but they accidentally rejected it. Making it harder to accidentally make the wrong decision in the first place, which is what #76 is about, will largely obviate this problem.
  3. Change CSOL-site to understand the concept of a revised review, and have it email learners if the mentor changed their original decision. This is somewhat in-line with our original design for the assessment tool prototype, in which we modeled the assessment process much like a github pull request--a conversation between mentor and learner that could be constantly updated with new information. Like a github pull request, it would only be irrevocably "done" when the learner acquired their badge--otherwise a rejected submission could always be re-opened by either participant.

For the short term, solution 2 obviously involves the least amount of work. :smile: In the longer term, I think our original prototype had the right idea.