Stats are calculated on a per-project basis (some exceptions apply)

jeffreywescott commented 8 years ago

(depends on #68)

Strategic Goals this issues impacts

!Learner stats match our subjective reality, elicit honest feedback, and promote self-directed, collaborative growth
!Game is balanced
Benefits
Teams are internally responsible for generating their own stats (not blocked by any external factor like moderator or reviewers) — agency :angel:!
All stakeholders can reason about stats more easily
Learner feedback loops are tightened
Players can see which projects still need retros (i.e. they have not earned stats from them yet)
Context

Cycles are a place for projects to be born, but they quickly mature and don't need any other parenting. they mature and die of their own accord.

Description

A project is the main object around which stats are generated. To make our game simpler, we want players to be in control of when a project is complete, to be incentivized to complete them, and to get immediate feedback upon project completion.

Mechanics Change

Stats are calculated and sent to learners as soon as sufficient data is collected, and not at /cycle end command from moderator (moderator is non-blocking)
Event trigger for stat calculation/notification is: all retros for project complete
Retros can only be submitted once per project per player, and no project stats cannot be seen by any player until every retro for the project has been completed
Distinguish between internal and external project reviews (see #68), and only use internal reviews for calculating stats like XP
Future external reviews will still cause update to review stats for both the team and the project (e.g. avg. completeness), but there will not be any modification to player-level stats earned from the project (e.g. XP)

tannerwelsh commented 8 years ago

moving to project-based (cycle-agnostic) stat calculation: need to answer "what is the minimum data needed to calculate stats and inform players?"

tannerwelsh commented 8 years ago

moving to review, would love feedback from @LearnersGuild/los. note update to Playbook

bundacia commented 8 years ago

I'm a little confused on a few points:

stats calculated and sent to learners as soon as sufficient data is collected (retros + project reviews), i.e. not at /cycle end command from moderator (moderator is non-blocking)

That's how it works already (except that we don't wait for project reviews because project review data isn't used for stats yet; Is this issue dependent on another issue that has us using that project review data?)

future [project] reviews (beyond N) will still cause update to stats, but there will not be a new notification.

Currently there is a notification in the project channel every time a project review is completed. Are we saying we want to remove that?

cycles are a place for projects to be born, but they quickly mature and don't need any other parenting. they mature and die of their own accord.

I'm not sure I agree that we need to (or should) divorce projects from cycles like this. Are any of these assumptions going away?

Team retrospectives happen every cycle, not once a project
Project teams can change every cycle
Project creation and retrospectives always happen during a specific cycle phases

Projects don't need to be too tightly coupled to cycles, but currently there is a specific cycle phase during which cycles may be born (the transition from GOAL_SELECTION to PRACTICE), and I think it's reasonable to have a specific cycle state during which project may be reviewed and end. Are we just trying to say that we want projects to be able to span multiple cycles?

This feels a bit like a proposal but the original tension isn't included here so I'm not sure what problem we're trying to solve. Maybe a user story would be helpful here.

I have a feeling that we're working rather hard to accommodate the "DM stats to users" UX we have now. Maybe things would be a lot simpler if stats were a pull, not a push. Just give the players a URL where they can see their up-to-date stats (and maybe even a command that slides that page out in a panel). Then notify them whenever events take place that could affect those stats (which we mostly already do), and they can go look at them. We could even make that page auto-update so you can keep it open during cycle reflection and watch your stats update in real time.

tannerwelsh commented 8 years ago

That's how it works already (except that we don't wait for project reviews because project review data isn't used for stats yet; Is this issue dependent on another issue that has us using that project review data?)

XP depends on project completeness and quality stats, so to publish XP we'd need to hold off on stat calculation until sufficient project reviews have been completed.

Currently there is a notification in the project channel every time a project review is completed. Are we saying we want to remove that?

We're talking about two different notifications. The stat update notification includes player-specific information about what stats were gained from the most recent project, and is not re-sent after a new project review (beyond the N needed for calculation) is submitted. The "new project review" notification doesn't include any stat data, and is unaffected by this issue.

I'm not sure I agree that we need to (or should) divorce projects from cycles like this. Are any of these assumptions going away?

Team retrospectives happen every cycle, not once a project Project teams can change every cycle Project creation and retrospectives always happen during a specific cycle phases

Yes. New assumptions we want to be moving towards:

Team retrospectives happen every project. No dependency on cycles.
Project teams can change for different projects.
Project formation happens during a specific phase, but a retrospective can happen at any time.

AFAIK, we want to move away from the notion of a cycle as a construct. Really we have a project formation process that moderators host on a regular basis, and in which players can participate in order to join projects that will match them with other players of similar skill level and at least one advanced player.

When these projects end (including retrospective and review) is irrelevant to the project formation process - it is the prerogative of the team to do so.

tannerwelsh commented 8 years ago

This feels a bit like a proposal but the original tension isn't included here so I'm not sure what problem we're trying to solve. Maybe a user story would be helpful here.

Part of the initiative to allow for pick-up projects (can go into more detail later, most appropriately at next week's LOS Road Map Sync).

Some user stories that this is supporting:

As a player, I don't want to be restrained by a moderator in how I form and end projects.
As a player, I want to start and work on projects of varying lengths.
As a player, I want a moderated project formation event to help me join projects where I'll be able to benefit from cognitive apprenticeship.
As a moderator, I don't want to have project creation constrained by anything other than the people who show up for voting + project formation. In other words, the state of the previous cycle shouldn't block the behavior of the current one.

shereefb commented 8 years ago

RFI /cc @LearnersGuild/software

jeffreywescott commented 8 years ago

Hey, all. I was going to move this to the implementation board, but before I do, I want to make sure I understand what is actually being delivered here.

From reading the description and the comments, I'm wondering if there's a hidden implication for some major data model changes to support this (unless I'm reading things wrong):

Are we removing cycles as a construct right now?
Are we trying to support multiple retrospectives per player per project?

Or, are we leaving those things the way they are right now and just changing when stats are calculated and when the project-stats notification gets sent, acknowledging that the notification received may immediately go out of date as a result of further project reviews?

shereefb commented 8 years ago

We are not removing cycles, or supporting multiple retros per player per project We are just chipping away at the tight coupling between projects and cycles. Moving towards a world were cycles are just a great way to co-ordinate a bunch of projects starting at the same time and sharing resources.

are we leaving those things the way they are right now and just changing when stats are calculated and when the project-stats notification gets sent,

yes.

acknowledging that the notification received may immediately go out of date as a result of further project reviews?

yep

jeffreywescott commented 8 years ago

Got it. Will move this over soon. 👍

shereefb commented 8 years ago

related #68

jeffreywescott commented 8 years ago

@shereefb -- do we want to move this over to implementation, or just wait for #68? Seems like we'll be duplicating some effort, but if this is high-value, we can certainly move it across. Fair-warning, though: there's a pretty big backlog of stuff, and we're down one engineer. :-(

shereefb commented 8 years ago

@tannerwelsh moving back to game mechanics because #68 really impacts this and gets rid of project review thresholds

tannerwelsh commented 8 years ago

updated description

tannerwelsh commented 8 years ago

both sub-issues in review, ready for review @shereefb @LearnersGuild/software

shereefb commented 8 years ago

hmmm... couple of things here that are bugging me

As a player, can I submit my retro after a cycle ends?
As a player, am I screwed if one of my team mates checks out and doesn't complete their retro? Or do all retros get auto-calculated at cycle end? Or do we decouple projects from cycles at cycle start, and then there's a way for a moderator to force-calculate a retro where all players haven't submitted it??

shereefb commented 8 years ago

Woah, just realized a great side-benefit of this is that we're now free of retaliatory-stats, since once all retros are in you can't change your retro. Right? Or would I still be able to change my retro because the cycle is still open? Something is not right here. This issue can't be implemented without retros being completely decoupled from cycles.

tannerwelsh commented 8 years ago

@shereefb good points. Added this to description:

Retros can only be submitted once per project per player, and no project stats cannot be seen by any player until every retro for the project has been completed

tannerwelsh commented 8 years ago

re: @shereefb

As a player, can I submit my retro after a cycle ends?

Sure. No such thing as a "cycle end" in this scenario. A new cycle starting doesn't force stat calculation, or prevent retros from being submitted for projects in REFLECTION state.

As a player, am I screwed if one of my team mates checks out and doesn't complete their retro?

Yes. Moderator could override by removing the player from the project. But otherwise teams are responsible for submitting their retros, and will not earn stats from any project until ALL retros are complete.

It shifts the authority of stat generation from moderator (via /cycle end) to teams (via submitting retros).

Or would I still be able to change my retro because the cycle is still open?

Nope. Retros can only be submitted once per player per project.

jeffreywescott commented 8 years ago

Just want to note here that currently, the retrospective UI doesn't do any client-side validation, nor does it have a way to "go back", and IIRC, many players have re-issued the /retro command to correct mistakes that they made entering data in the past. I think we must address those UI issues before we can make it so that retros can only be submitted once per project.

jeffreywescott commented 8 years ago

@prattsj -- am I right about validation / "going back" above ^^^?

heyheyjp commented 8 years ago

Correct - currently no back navigation. The trickiest part about back nav here is supporting page-by-page response persistence in a situation when we don't know up front how many "pages" there are (otherwise, we could just hold all responses client-side and send at the end, in which case we wouldn't need to call the server to retrieve page-specific values previously submitted, making back navigation - and prepopulating previously submitted answers -- relatively easy).

tannerwelsh commented 8 years ago

Opened a new issue to cover this. Wasn't aware of the blocking navigation piece.

jeffreywescott commented 8 years ago

Related #83

jeffreywescott commented 8 years ago

Issue moved to LearnersGuild/game #502 via ZenHub

LearnersGuild / game-prototype