Incorporate internal project review to retrospective

tannerwelsh commented 8 years ago

Strategic Goals this issues impacts

!Learner stats match our subjective reality, elicit honest feedback, and promote self-directed, collaborative growth
!Game is balanced
Benefits
All stakeholders can reason about stats more easily
Learner feedback loops are tightened
Context

Most of the stats we have now rely upon data collected in retro, not in project review. However, because XP depends upon both, we have an awkward situation where no stats can be calculated until a project has all retros completed as well as N reviews completed.

Description

To improve this, we'll include an internal project review in the retrospective. This way, each player will evaluate completeness and quality as part of the retro.

What we gain is that a project can be deemed "complete" as soon as every retro is complete. This drastically simplifies the logic of #20, making the entire process easier to reason about for all stakeholders.

Mechanics Change

[ ] Include project review (completeness / quality) in the retro survey
[ ] Distinguish between "internal reviews" and "external reviews" for a project, so that we can use only "internal reviews" when calculating stats like XP
UI Design

Move the following questions into the retrospective survey:

- What percentage of this project has been completed?
- How would you rate the quality of this project on a scale of 0-100%?

Both are required questions, asked once.

Position: after the contribution question.

Update the behavior of `/review` command:

When a player enters /review #project-name AND they are a member of the project provided, respond with the following message:

Whoops! Looks like you are on team #project-name. If you'd like to review your project, you can do so with `/retro`.

Include internal review scores in post-retro DM

**RETROSPECTIVE RESULTS:** #${project.name}

**Internal project reviews:**
Completeness: ${project.avgInternalCompleteness} (avg of: ${project.internalCompletenessReviews.join(', ')})
Quality: ${project.avgInternalQuality} (avg of: ${project.internalQualityReviews.join(', ')})

**Feedback from your team:**
${teamFeedbackList.join('  \n\n')}

**Hours contributed:**
Team size: ${team.length}
Your hours: ${stats.hours || 0}
All team hours: ${stats.teamHours || 0}

${teamHoursList.join('  \n')}

**Contribution to the project:**
Self-assessed: ${stats.rcSelf || 0}%
Team-assessed: ${stats.rcOther || 0}%

Your estimated contribution to the project: ${stats.rc || 0}%
Expected contribution for # of hours: ${stats.ec || 0}%
Contribution difference: ${stats.ecd || 0}%

**Stats earned for this project:**
Culture Contribution: ${stats.cc || 0}%
Team Play: ${stats.tp || 0}%

tannerwelsh commented 8 years ago

@shereefb thoughts? let's discuss this tomorrow. It impacts #21

shereefb commented 8 years ago

Another way to unblock pure retro-based stats is to calculate them after the retro, and calculate XP after the project reviews come in.

XP gained can be communicated in the project channel along with the project completeness and quality

related #20 related #19

shereefb commented 8 years ago

related #56

tannerwelsh commented 8 years ago

Updated description

tannerwelsh commented 8 years ago

updated description with UI

tannerwelsh commented 8 years ago

ready for review @shereefb + @LearnersGuild/software

tannerwelsh commented 8 years ago

ugh. i hate updating descriptions with each new iteration. just pasted over the previous version with something else. :( clipboard history, please help me.

tannerwelsh commented 8 years ago

ok. back in business.

shereefb commented 8 years ago

FANtastic. Love it. Ship it. Do it. Yeah!

jrob8577 commented 8 years ago

Doesn't this increase the amount of time needed for retros? This feels like it's tightly coupling the review to the retro, which means that some folks inputs will be more rushed (less informed).

tannerwelsh commented 8 years ago

re: @jrob8577

Doesn't this increase the amount of time needed for retros?

Yes, it probably will. More of the reflection time will be dedicated to retros, and whatever is leftover to external reviews (of other people's projects). Definitely puts more strain on adv. players on multiple teams.

This feels like it's tightly coupling the review to the retro, which means that some folks inputs will be more rushed (less informed).

It is tightly coupling for sure. Can't we fix the rushing piece if we allow more time for retros? I think it's worth the benefit incurred here.

jrob8577 commented 8 years ago

@tannerwelsh Allowing more time for retros makes sense, but then takes more time from project development. 😄 This may be a non issue if the advanced players (there were issues raised about this term in home group, but I'm not sure what other term to use to describe the ... more experienced/higher elo? player on a team) are on less teams as we move forward.

tannerwelsh commented 8 years ago

Allowing more time for retros makes sense, but then takes more time from project development.

the grand tradeoff: reflection vs. action. :) might even call it a dynamic tension. yeah, i don't have any answers, just proposals for modifications.

and yes, the intent is certainly to have advanced players on fewer teams. this should be more feasible as the skill diversity of a chapter increases.

jeffreywescott commented 8 years ago

@bundacia -- are there any implementation complexities with having these "review" questions be a part of the retrospective survey?

jeffreywescott commented 8 years ago

Waiting for @bundacia to review before moving to implementation board. This also blocks #21 from moving across.

bundacia commented 8 years ago

I believe we already have questions where the subject is the project, so there's no implementation risk in that area that I can see. I have a little concern about only using internal reviews for calculating stats. The team's opinion about quality and completeness is likely to be the most biased and in general the number of people your sampling for those numbers goes way down as well.

jeffreywescott commented 8 years ago

Thanks, @bundacia.

Good point on the reviews -- I had the same issue. From what I understand, though, external reviews will (eventually) be used to "keep people honest", as there will be a stat for how far off internal reviews are from external reviews.

tannerwelsh commented 8 years ago

The team's opinion about quality and completeness is likely to be the most biased and in general the number of people your sampling for those numbers goes way down as well.

This is a good point @bundacia, and we're planning to address it with #71 (review bias stat)

tannerwelsh commented 8 years ago

@jeffreywescott can this be moved to engineering board?

jeffreywescott commented 8 years ago

Issue moved to LearnersGuild/game #501 via ZenHub

LearnersGuild / game-prototype