Declaring 0 hours, and being "removed from a team" should be the same thing.

[shereefb]

Critical goal Served:

!Learner stats match our subjective reality, elicit honest feedback, and promote self-directed, collaborative growth

Description

As a learner, if I end up not working on a team (say I was sick), then my stats shouldn't be impacted.

New Logic

If in my retro I declare 0 hours, then the project should not impact my stats. Regardless of what my teammates declare about me in their retros.

UI

When declaring hours, if the player enters "0", a validation message indicates that they won't receive stats:

Declaring 0 hours for this project is the same as not being on the project. You will not earn any stats from this project.

[shereefb]

@tannerwelsh please review and move to RFI

[jeffreywescott]

To me, it looks like this needs a review from Software Engineering, since only they know how stats are calculated, and whether this raises issues that will be difficult to address in implementation.

Please remember that the Review stage is where all stakeholders have a chance to provide input (this is to avoid back-and-forth thrash between design and implementation boards).

[shereefb]

/cc @LearnersGuild/software please review

[heyheyjp]

@shereefb: I'm not sure I've ever seen an instance where someone declared 0 hours, but this makes sense for that case. What we have seen is cases where someone was absent on review day and either did or didn't contribute to the project. In the latter case, again, this change applies easily enough. Though they're a member on the team, if everyone gives them a 0 in contribution, we can consider this to be the same as them explicitly declaring 0 hours.

Where things get tricky is when someone either reports 0 hours or doesn't report any hours at all and is given a non-0 contribution score by a teammate. How should the system behave in this scenario?

Note that because we don't know that a player on a project team hasn't contributed to the project until they either (A) report 0 hours in the retro survey or (B) stats are processed without the player submitting a survey at all (in which case we actually can't safely assume that they didn't contribute)...each of their teammates are still going to be asked questions about them in the survey. So this introduces the possibility that someone would give feedback about a subject that is somehow inconsistent with the subject's own reporting of their involvement. I suppose that in the event that someone reports 0 hours (or doesn't report at all), we could void all stats-related responses about them from their teammates.

In summary, the cleanest way to handle this in the short term implementation-wise would be to simply ensure that people who aren't actually expected to contribute to a project (and therefore for whom we don't want to see any change in stats) are removed from the project team before the cycle/project enters REFLECTION state. But it shouldn't be terribly expensive to try to address this in how the stats are calculated, either. It just feels a lot less clean.

[tannerwelsh]

ensure that people who aren't actually expected to contribute to a project (and therefore for whom we don't want to see any change in stats) are removed from the project team before the cycle/project enters REFLECTION state

👍

second-choice vote would be:

1. if player gets 0% contribution from rest of team
   • AND
     • they submitted a retro with 0 hours OR didn't submit a retro for the project
   • THEN mark them as not being on the team
2. if player gets > 0% contribution from rest of team
   • AND they didn't submit a retro for the project
   • THEN block project stat calculation for project until player completes retro
     • and notify the team
3. if player gets >= 0% contribution from rest of team
   • AND they submitted a retro with > 0 hours
   • THEN score them as usual

[jeffreywescott]

if player gets > 0% contribution from rest of team

• AND they didn't submit a retro for the project
• THEN block project stat calculation for project until player completes retro
  • and notify the team

We can't really "block" stat calculation, but it doesn't happen until all team retros are complete anyway.

So are you saying we just need another notification to the players who have completed the retro saying that a certain other player has not completed it? What should the notification say?

Moving this back to UI Design for now.

[tannerwelsh]

Thanks for the questions @jeffreywescott, but I don't think they're blocking. I updated the proposal in the top comment for what this issue is really meant to solve - my later comments are more a response to @prattsj's comments and some brainstorming about how a more complete solution would look.

[tannerwelsh]

Ready for (re-) review @LearnersGuild/software (note that the proposal is what is in the top comment only, not the thread afterwards)

[bundacia]

This introduces a possible loophole.

Imagine I've been a pain to work with all week and I'm not pulling my weight. I get the sense that my team is frustrated with me and will probably give me poor reviews. So to protect my stats I just declare "0" hours. I loose the chance to gain a little XP, but I don't take the hit on my ELO or any other stats that I should.

Maybe we just need a simple interface that allows a moderator to remove people from teams (we've already got a project/team management interface in the pipeline). Then we put the onus on the player or team to contact the moderator before reflection if someone needs to be removed. I think that would solve the problem for now.

Down the road, if we run into scaling issues with that approach we could create a new role filled by more advanced learners that gives them access to that interface so that it's not always an LG employee that has to manage requests to remove people from teams. We could also create an interface for learners to submit a request to remove a player (including themselves) from a team.

[tannerwelsh]

Good points @bundacia. I think we can handle that loophole with better admin features in the future, but for now this change seems safe enough to try IMHO.

[bundacia]

@tannerwelsh, can you elaborate on how better admin features would plug the loophole?

[tannerwelsh]

What you said @bundacia - if a moderator and/or players could manage teams, then we wouldn't need the "zero hours" rule.

[bundacia]

@tannerwelsh: ok, in that case this may not be needed. @prattsj is already working on an admin interface for managing teams that would likely be done before this. @prattsj, does the admin work you've been doing provide a way for a moderator to remove someone from a team?

[heyheyjp]

@bundacia: yep, includes the ability to both remove and add players to a project team.

[heyheyjp]

if player gets > 0% contribution from rest of team AND they didn't submit a retro for the project THEN block project stat calculation for project until player completes retro and notify the team

@tannerwelsh: my initial gut reaction to this was that it seems pretty risky and unfair to those players left hanging. however, after thinking through a number of alternatives of varying levels of complexity, I'm pretty OK with this as a resolution for now with one modification: if a player must leave a project before it is completed, we might consider having a rule that that player forfeits any "credit" for their involvement in the project and will be removed from the project team immediately. This, of course, unblocks the rest of the players from getting their feedback & stats adjustments at a time when the surveys haven't yet been generated to include the missing player. To apply pressure that discourages people from using this as a way to avoid feedback about poor involvement on a project, we can have a threshold for number of times a player has been absent from a project over some period of time - too few and it puts you in the danger zone.

WDYT?

[shereefb]

@prattsj sometimes people get sick, or have an emergency. Why punish them for that? I think this adds more complexity and isn't worth it.

[heyheyjp]

@shereefb I don't disagree that it's less than optimal, but right now if that person doesn't manage to submit a retro before the project is "closed"/cycle ends, they never have an opportunity to do it again. Is it more fair that their teammates then never get their feedback and stats updates?

[tannerwelsh]

If we have the admin tool to add/remove team members before retro time, then is this issue moot @shereefb?

[tannerwelsh]

my initial gut reaction to this was that it seems pretty risky and unfair to those players left hanging. however, after thinking through a number of alternatives of varying levels of complexity, I'm pretty OK with this as a resolution for now with one modification: if a player must leave a project before it is completed, we might consider having a rule that that player forfeits any "credit" for their involvement in the project and will be removed from the project team immediately.

Yeah @prattsj I'm afraid we're in some mucky territory here. I have felt similarly, and haven't yet come across a solution that is absolutely fair to everyone equally.

To give you a glimpse into what I'm thinking for a longer-term solution (probably not v2, since the implementation implications are significant): I'd like to remove "statefulness" from cycles, and move it instead to projects. So any project would either be IN PROGRESS or COMPLETE. To be COMPLETE, every team member has to submit their retro & internal review. Only completed projects earn stats for their team.

Then, to incentivize players actually completing projects, we could introduce something like a limit to how many projects you can have as IN PROGRESS for a player who hasn't yet submitted their retro for these projects.

With this, we can accommodate cases where a player is absent for part of their project - they just submit their retro when they get back. The rest of their team has to wait a little longer to earn stats, but they will eventually earn them. It's not a matter of if, but when.

These changes would require some tinkering with the "red zone" mechanics, but nothing unmanageable.

[shereefb]

+1000 to decoupling cycle state from project state Really, where we heading: the cycle is really just a co-ordination event that happens to kick off a bunch of projects at the same time and distribute resources amongst them. it should be /cycle init, then /cycle launch. That's it. Then projects can individually move into /project reflect

[shereefb]

Tracking this decoupling here: #114

[tannerwelsh]

Moving to RFI for the "0 hours = no stats" piece. Other good work raised here tracked in other issues (i.e. team admin tools, cycle decoupling)

[jeffreywescott]

Issue moved to LearnersGuild/game #573 via ZenHub