Add roles & levels

[tannerwelsh]

Benefits

What are the benefits of this change, and whom do they impact?

• Learners are incentivized to stretch; there are rewards for progression
• Learners are engaged in multiple roles for learning
• Learners have access to Coaches and Project Managers for assistance and guidance

Description

Describe the change, and provide any needed context.

Introduce new roles and levels to give more structure to the game.

See sub-issues #122 #126 and https://github.com/LearnersGuild/playbook/issues/75 for work and more details.

[tannerwelsh]

Ready for review @LearnersGuild/software. Please break feedback up into two parts:

1. Feedback on the non-software-dependent changes (i.e. do these mechanics seem OK?)
2. Feedback on the software implications

We'd like to start by formalizing the roles/levels in the playbook & Player Support, see how it goes, and then move to support it w/ software. Could be that the software piece becomes its own issue if combining the two here is too much.

[bundacia]

The game mechanics of this seem to make sense to me. I have a couple of questions:

1. Can we use a static ELO thresholds to enforce a level?

Since ELO is just a relative ranking it seems odd to me to see a hard coded ELO threshold for these levels. It's my understanding that my ELO can go down even though my skill level has not, just because other people are advancing faster than me. So I could be in a situation where I get knocked back a level just because my ELO took that kind of a hit. Is there a more flexible way to incorporate ELO?

2. How does this relate to "pools"

Are we thinking that each level would be a voting pool and we might augment the pool feature to allow moderators to restrict what type of goals can be voted on in each pool? We don't really need to answer this yet, just curious. And it may inform the way we build pools.

[tannerwelsh]

Good questions @bundacia.

Can we use a static ELO thresholds to enforce a level?

I think it's safe enough to try, and here's why:

1. If others are advancing faster than you (i.e. they are becoming more effective faster than you), their Elo will climb and yours will drop when you play. So in a sense, you do need to "catch up" with your peers to stay in a level. But this is not as bad as it sounds because as their Elo increases, you can "lose" to them and still gain Elo, if you did better than expected.
2. There will be more players crossing the level threshold, so you shouldn't stay at "the bottom of the pack" for too long, as there will be a constant flow.
3. Because we're using the same "in the red" mechanics for dropping down a level, if your Elo drops you are not immediately bumped back. It has to be below the threshold and decline for 2 weeks in order for you to "bump back" a level. This gives you time to recover, and will make going back a level much less frequent than going forward.*

* quick note on game design: "game over" events should happen a lot less frequently than "level up" events. Building momentum and progression should be the norm; regression is more rare.

How does this relate to "pools"

Each level would map to a "pool" for goal selection. Players choose on goals in their pool. To start, this will be softly enforced, but eventually it'd be strictly enforced, i.e. players can't work on goals outside of their level.

[bundacia]

Thanks for the explanation @tannerwelsh! Sounds safe enough to try!

[jrob8577]

I'm holding a tension around the coach and PM roles being accessible to learners at the moment. Those roles have the potential for such a high impact on the learning community at large, that it feels like a really big step from where we're at to allowing anyone to move into that role. Wasn't there a requirement of approval from SEPs at one point, or am I mis-remembering that?

The point about levels being cumulative, and granting privileges of previous levels - does that also apply to requirements? If someone in a coach role (? this is probably a bad example) has estimation bias outside of the 8% range in successive weeks, does this mean they move from Level 3 back to Level 1?

I like the coach rotation being built into this model - I definitely feel like I'm spending more time managing than developing, and look forward to the change that coaching would allow. This may also be the result of an uneven distribution of learners on OSS projects (with 5 teams, I must spend more time managing than pairing, for example).

The 1080 ELO for working on LOS is interesting - is there anything informing this number? Will LOS take on responsibility for the learner's growth (and all of the other accountabilities that are currently held by roles in the Player Support team)? Is this some shared accountability/role between Player Support and LOS? Do coaches, PMs, and LFs support players that are playing with the LOS team? Will the LOS team be on site/available to support multiple players (or will the algorithm be limiting the number of folks attached to LOS projects)?

[deonna]

I'm curious about what happened to the requirement that at least 2 coaches vouch for an aspiring coach? And that at least 2 PMs vouch for an aspiring PM?

Holding a tension around these (IMO, big) changes being introduced after the Player Support Tactical on Wednesday, because I was a 👍 on the changes introduced by this issue after my concerns were addressed during Tactical, but noticed the above requirement was changed only after @jrob8577 pointed it out in his comment.

I liked the requirement of >= 2 coaches/PMs vouching for folks because it added some assurance that we wouldn't be introducing coaches that would have a negative cultural impact on the space. I'd like to know how coach and PM stats will be defined, because that piece seems crucial to resolving my cultural impact tension.

Overall, I think this is a great roadmap for learner growth.

[carlabagdonas]

Excited to try these changes. A few questions:

1. It seems to remove the distinction / voting pool separation between foundational projects and practice goals. So far it seems like this distinction has been really helpful and that the leap from 0 (foundations) to 1 (practice goals) is pretty significant. I'm holding a tension around people who really need to be working on foundational goals getting put on practice projects and being way outside their ZPD, and thus wasting time. If we could better preserve this distinction, I'd feel better.
2. Interesting thought to have cultural stat thresholds as requirements for advancing to higher technical levels. Will we encounter cases of people who are really technically advanced / prepared to work on OSS projects but are being held back because of their health stat scores? I think this will be a pain point, but also appreciate the upward pressure it puts on health stats for people on the lower end of the spectrum. Safe to try, but I anticipate some sticky places.
3. I hold a somewhat similar tension around learners serving as Coaches and PMs, but mostly from the perspective of making sure they're spending the majority of their time in their ZPD. They should only coach a max of 1 week per 4-5 cycles. I think the XP thresholds will make it so that they can't actually be PMs until much later in the program, so I have less of a tension around that one, except to wonder whether it's in their best interests, learning-wise to be spending time on project management instead of coding. I'll leave it to people who are smarter than me to make that call, though :)

@tannerwelsh I think maybe this was a typo? "Est. bias less than -8% or greater than +8%" (should less than and greater than be swapped? or am I misreading...?)

[jrob8577]

@carlabagdonas you reminded me that one of my tensions was around the learners as PMs not leading to the end goal that we have defined: kick ass web engineer. Just wanted to weigh in on that as well.

[deonna]

Thinking about @carlabagdonas and @jrob8577's concerns about PM-ing vs. coding a little more. They're making me question my assumptions about the PM role.

I'm imagining PMs (in addition to the 3 accountabilities defined in the original issue):

• Figuring out how to abstract a large, multi-week project so it is well-defined in the user stories + specs they write
• Doing initial setup and configuration of the project they're managing
• Making architecture and design decisions within the codebase

To me, all of these seem important to becoming a kickass web engineer, because they require learners to have validated opinions on all of these issues and to have developed the programming expertise to make smart decisions based on those opinions.

Am I missing something? :#

[jrob8577]

@deonna It seems to me like these are very aggressive goals for folks that are coming in with very little experience to be able to manage by the end of a ten month period. I am more focused on guiding learners towards being able to create a standalone web application, similar to something they would need to do as a take home project for an interview, and be able to understand the fundamentals involved that process (what is a route, how does it relate to HTTP, what's the difference between building an API and not, different CSS approaches, basic OOP, etc.).

Thinking of my management roles, when I was looking for junior engineers for my teams, I was looking for three things:

1. Can they contribute to building our web sites
2. Can they work effectively in a team
3. Do they know when to ask for help

Knowing how to decompose a product into user stories feels to me like a product management role/responsibility, not an engineering responsibility. Most of the translation of features into technical specs was managed by my senior engineers, who were also tasked with communicating those tasks, and why they were broken down in a certain way, to the junior engineers.

Initial setup and configuration I feel like will only be important for the interview style coding challenges - most folks, I imagine, will not be working on a greenfield project, so need experience diving into, and making sense of, an existing codebase, rather than creating a new codebase (I'm regretting the lack of codebase this week in lizardboard; I feel like it really stifled the amount of work we could otherwise have done).

Architecture and design decisions - I think this is important at the relevant scale, but systems architecture experience is not something I anticipate learners walking out of here with. It feels like we're trying to overshoot our goals with some of these.

To your final point, I totally agree that having an opinion and expertise in this realm is critical, but I don't think we're creating product managers or systems architects. I think its great to have these conversations with learners so that they are exposed to the thinking behind systems architecture, but the PM role continues to feel to me like it would be changing the focus of the learners from development to product and management.

[bundacia]

Regarding having learners contribute to LOS core, I think we need to make sure there's some mechanism to limit the number of people attempting to contribute at once. Like maybe we have a fixed number of openings that people can cycle through.

[shereefb]

Wasn't there a requirement of approval from SEPs at one point, or am I mis-remembering that?

There was. We got rid of it, and we got rid of PMs doing interviews. Basically getting rid of all "human gatekeeping" and moving to "stats gatekeeping" is the trend. Human gatekeeping is expensive for relationships, and doesn't scale. For example, Jared has been talking about how people on tresello aren't exposing their ignorance for fear of "the interview"

[shereefb]

It seems to me like these are very aggressive goals for folks that are coming in with very little experience to be able to manage by the end of a ten month period

They are! I think a very small percentage of our learners will get there in 40 weeks.

[deonna]

@jrob8577 your rationale makes a lot of sense - thanks for the detailed explanation.

That said, I do like the idea of leaving the PM option open as a stretchy goal that only a small percentage of learners may achieve. Without it, I feel as if we are artificially limiting learners' growth by putting a cap on the heights we think they can realistically achieve in 40 weeks.

If a learner is producing code that shows a strong attention to best practices, maintainability, good design, considered architectural decisions, etc., then I think there should be a role for them, and PM (as defined right now) feels like a good fit.

[shereefb]

Regarding having learners contribute to LOS core, I think we need to make sure there's some mechanism to limit the number of people attempting to contribute at once. Like maybe we have a fixed number of openings that people can cycle through.

+1 to that. It's ultimately LL of LOS call who temporarily joins the team. Think of it as free junior engineers.

[jaredatron]

These changes seem to add "project management" skills to our implicit curriculum. Is that really the best use of our learners time? I doubt very many employers are looking for project management in junior engineers. My gut feeling is that we can train awesome junior engineers without putting them in any PM roles at LG.

Basically I agree with everything @jrob8577 said :P

[tannerwelsh]

Thanks for all the great feedback folks. FYI, I've added two sub-issues for this epic to be complete: #122 and #126

For @jrob8577 @deadlyicon: hear what y'all are saying about PM'ing being to much of an "extra" skill. At the moment, my line of thinking is more or less what @deonna mentioned:

Without it, I feel as if we are artificially limiting learners' growth by putting a cap on the heights we think they can realistically achieve in 40 weeks.

So in the interest of harvesting our collective intelligence, is there a better role that you'd like to set up as something that learners should aspire to?

Think of it this way: whatever our "levels" are, I don't want them to ever send the message that we don't expect people to surprise us and to achieve great things even in short amounts of time. If we were a program for learning to play basketball, I'd want our highest level to be the equivalent of "1st round NBA draft pick", not "coach of the high school varsity team".

So, my question for y'all (and especially the pro players): what is an appropriate and inspiring progression for learners to aspire to?

What benefits and abilities should learners who learn excellently receive? What roles should they be endowed with, so that they can continue to refine their skills and face new and interesting challenges?

[tannerwelsh]

To be clear, we should not be saying "you've failed if you aren't a superstar". Help people set realistic expectations and celebrate their achievements when they do so. But let's always leave room for people to grow beyond their/our expectations.

[jrob8577]

It's not so much about better role in my mind, as focusing our learning outcomes and goals. If the goal is to become a kick ass web engineer, then stepping into a PM role feels orthogonal to that goal - my experience as PM on floworky and lizardboard is that I spend a significant amount of time managing, and very little time doing technical work (including technical management).

In terms of progression, folks should be more experienced technical leaders on a team. Is this what we're trying to capture with the PM role as it stands? (I am holding a tension around terms like lead or senior, since this sets expectations that are not consistent with where I imagine folks will be starting when getting a role in industry.)

[tannerwelsh]

Thanks for the clarification @jrob8577. I agree that we shouldn't be creating "watered-down" versions of "lead" or "senior" if learners could not actually get lead/senior positions in the industry. Definitely don't want to create a false sense of achievement/qualification that is asymmetrical to an actual qualification level.

Also good to know that your PM time is mostly managing, and not too technical. If you don't think that the experience you're having would be useful for learners, then I have two follow-up concerns:

1. What is a role that carries added responsibility and is earned through a demonstration of advanced skill, but one in which the challenge accelerates learning towards the overall goal rather than adds friction or diverts their growth trajectory?
2. How do we make sure that you (and other pro players, speaking somewhat self-interestedly here) are able to spend time in roles that allow you to learn, instead of just managing?

[tannerwelsh]

In terms of progression, folks should be more experienced technical leaders on a team. Is this what we're trying to capture with the PM role as it stands?

I think this issue is confusing because it is trying to do several things at once. Let me try to decompose it as best I can.

Two of the trajectories that I personally believe we should be architecting for learners are the following:

• An arc from scaffolded structure towards informed agency
• An arc from personal development towards community leadership

In other words, we are saying:

When you start, you have few options. This limitation is actually a benefit, because it allows you to focus intensely on your own development: establishing good learning habits, solidifying foundational technical and collaborative skills, developing a fluency with the language and norms of our learning environment. You advance by taking on more difficult and challenging work. Push yourself to learn beyond your comfort and capacity, integrate those learnings, and repeat. There will always be greater and more interesting relevant challenges for you to face. As you advance, you earn more freedom to choose how to spend your time, what skills to focus on, and what work to pursue. With this status comes the accountability (and privilege) to mentor and aid others, to distribute and compund your learnings throughout the community.

So in my view a role earned by players who advance several levels should reflect this arc: it should offer them a chance to tackle new and interesting (and always relevant) challenges, it should give them new ways to take control of their learning, and it should confer greater responsibility to contribute to the learning collective.

From your observations, it sounds like the biggest question is in regard to the relevancy feature of the challenge of being a PM, and to a lesser degree the value to the community question. I'd love to brainstorm ways to improve the PM role, or replace it with something more appropriate.

[carlabagdonas]

This is a great conversation - glad I came back in to catch up on it :)

@tannerwelsh I really like the framing & question you're asking here:

Think of it this way: whatever our "levels" are, I don't want them to ever send the message that we don't expect people to surprise us and to achieve great things even in short amounts of time. If we were a program for learning to play basketball, I'd want our highest level to be the equivalent of "1st round NBA draft pick", not "coach of the high school varsity team".

So, my question for y'all (and especially the pro players): what is an appropriate and inspiring progression for learners to aspire to?

Don't have the answer but wanna give a ++ to the question.

[carlabagdonas]

Separately, here's a little tidbit I was wondering about:

If my Elo fluctuates back and forth on the cusp of a level, will I constantly be bumping back and forth between levels (and therefore the types of projects available to me?) That seems like a heavy cognitive load for a learner in terms of context switching. I thought I remembered @shereefb saying something about a kind of "probationary" or "red zone" for each level - basically I think it would be good for there to be a little buffer zone, or for the stats thresholds to overlap a bit between levels (or maybe once you attain a particular level you can't drop back down even if your Elo fluctuates, but maybe if your health stats do...)

[tannerwelsh]

Thanks for all the great feedback y'all. I've moved the final proposal for this to a PR on the playbook: https://github.com/LearnersGuild/playbook/pull/76. Would love any final feedback you want to offer there.

tl;dr of latest changes

• moved software-dependent part to own issue: #129
• changed "Coach" to "Technical Coach"
• changed "Project Manager" to "Project Lead"
• specified Technical Coach purpose and accountabilities (in Glassfrog and in Playbook)
• specified Project Lead purpose and accountabilities (in Glassfrog and in Playbook)
• clarified how players move up and back levels
• simplified how level requirements are presented
• simplified how level abilities are articulated
• clarified goal types

[jeffreywescott]

Do we need to figure team size into this? See: https://github.com/LearnersGuild/game/issues/447

[tannerwelsh]

@jeffreywescott team size seems related to this epic, but not critical for it to be implemented. Probs a separate issue, dependent on this epic closing.

[jeffreywescott]

Okay, for Q1 2017, we're aiming for this epic to be completed. Please add any associated stories to it in Clubhouse, then close this one.

[jeffreywescott]

Also, I suggest adding an Epic for roles in Clubhouse.

[tannerwelsh]

Moved to https://app.clubhouse.io/learnersguild/epic/226/in-game-roles