when creating projects, votes and ECC should influence player assignment

[tannerwelsh]

Use a simplified version of the original "project creation" algorithm: https://github.com/LearnersGuild/learning-os/blob/master/game/processes/project-creation.md

Constraints:

• Team leads can only lead projects from the same goal
• No project can exist without a more experienced team lead
• No team should be without a Lead
• Team Members can only be on one project
• Each Team Member's experience and skill level should be approximately equal

[tannerwelsh]

Algorithm idea:

• Find most popular goal (based on votes)
• Create 1 project of recommended size based on that goal
• Find players who voted for goal
• Assign team members in ASC order ECC who have voted for this goal
• Fill out the rest of the team members if there aren’t enough who voted for this goal
  • Start with players who voted for goals that weren’t selected
  • OR don’t have any votes left
• Assign team lead by:
  • Filter all eligible leads* from chapter players
  • If there is an eligible lead:
  • Try to find a player who is already a team lead on a project based on the same goal
  • Try to find a player who also voted for this goal
  • Otherwise, pick one and assign as team lead
  • Remove all players’ votes and rerun algorithm
  • Else
  • Remove this goal and all votes related to it and rerun algorithm

*Eligible leads =

• ECC at least (10 * no of teams) above mean of team members
• Not assigned to projects based on a different goal

[shereefb]

@jeffreywescott this is ready for backlog.

[jeffreywescott]

@tannerwelsh / @shereefb -- this algorithm seems pretty complicated for a first pass. I'm also not sure I understand all of it. Can we work it through here with some examples in the comments?

• Find most popular goal (based on votes)
• Create 1 project of recommended size based on that goal

Does the recommended team size include the team lead, or is the team lead an "extra" person on top of the recommended size?

• Find players who voted for goal
• Assign team members in ASC order ECC who have voted for this goal

Meaning, assign team members with the lowest ECC (least experience) first, right?

• Fill out the rest of the team members if there aren’t enough who voted for this goal
  • Start with players who voted for goals that weren’t selected

This is a bit confusing, because we're only selecting one goal at a time. How will we know which goals have been selected and which haven't?

• OR don’t have any votes left

This is also confusing. In theory, most or all people will be in this state, especially since the /vote command requires two votes to be sent at once.

• Assign team lead by:
  • Filter all eligible leads* from chapter players
  • If there is an eligible lead:
  • Try to find a player who is already a team lead on a project based on the same goal
  • Try to find a player who also voted for this goal
  • Otherwise, pick one and assign as team lead
  • Remove all players’ votes and rerun algorithm

I don't understand what this means -- remove all players votes? Remove the player's votes who was assigned as a team lead? Remove all players votes who are on the project?

• Else
  • Remove this goal and all votes related to it and rerun algorithm

I don't understand this bit about "removing votes".

*Eligible leads =

• ECC at least (10 * no of teams) above mean of team members

Assuming "mean of team members" excludes the team lead?

• Not assigned to projects based on a different goal

Examples

First cycle:

numPlayers = 17
playerECCs == [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 200, 200]  // last 2 are paid players
numTeams == 7
teamSizes == [2, 2, 2, 2, 2, 2, 3]
teamMeanECCs == [0, 0, 0, 0, 0, 0, 0]
// each paid player must be able to lead 3 or 4 teams
// OK: 200 > 40 (meanECC + 10 * 4)

[jeffreywescott]

@shereefb / @tannerwelsh:

I hacked-together a very rough DRAFT of something similar to Tanner's algorithm, along with some data generation for what week 1 will look like. It's ES2015, so you'll need babel-cli to run it.

The code is ugly as shit -- don't judge me. I just need something concrete to think this through:

https://gist.github.com/jeffreywescott/d04a4d1c92a4ffc657fe11832dcdda23

[tannerwelsh]

Thanks for the questions @jeffreywescott. It is messy. I'll do my best to clarify, and I'll take a stab at editing your first attempt at implementation as well.

Does the recommended team size include the team lead, or is the team lead an "extra" person on top of the recommended size?

Yes, rec'd size includes team lead. So team-size-2 is the smallest size possible.

Meaning, assign team members with the lowest ECC (least experience) first, right?

Correct.

This is a bit confusing, because we're only selecting one goal at a time. How will we know which goals have been selected and which haven't?

You're right. Improvement: select the first unassigned player who voted for the least popular goal

This is also confusing. In theory, most or all people will be in this state, especially since the /vote command requires two votes to be sent at once.

"Not having any votes left" is confusing, it's intended to mean those players whose votes have been removed by this algorithm, in the step Remove this goal and all votes related to it and rerun algorithm

I don't understand what this means -- remove all players votes? Remove the player's votes who was assigned as a team lead? Remove all players votes who are on the project?

Should be "From the players assigned to this project, remove their votes for this goal."

Assuming "mean of team members" excludes the team lead?

Yes.

I don't understand this bit about "removing votes".

This comes from the Single Transferable Vote system that we're basing this algorithm on: https://en.wikipedia.org/wiki/Single_transferable_vote

For the sake of the algorithm, we need a temporary, isolated, mutable "vote roster" that is independent from the players original vote data.

[jeffreywescott]

@tannerwelsh -- we should generate some more "test case data sets" and play with them in the prototype algorithm.

I added something to the algorithm that "removes outliers", both because your original description said "Each Team Member's experience and skill level should be approximately equal", and because I made the wrong assumption that team-size-N did not include the lead.

We should pair on it and get the prototype version refined. :-)

[tannerwelsh]

@jeffreywescott thanks for the update. I'll take a crack at the test cases (created issue), and then if we can pair later today or Monday on the algorithm that'd be great.

[shereefb]

@jeffreywescott / @tannerwelsh raising a flag that this algorithm will only get more and more complex from here on. It's really worth investing in great test cases, and very detailed comments/documentation. Both on the game/design spec side, and code side.

We need to be capturing the "why" we're doing something in there.

This team formation piece is going to make a very big difference in our players' experience and I suspect we'll be tweaking it a lot. Taking into consideration peer to peer feedback, the number of times people get to work on what they're voting for, and maybe even cross-chapter availability.

[tannerwelsh]

Ok some thoughts on project formation, to make this more manageable.

What if, for v0, we remove the team-lead assignment part?

In other words, team leads would abstain from voting and then a different process assigns them after the fact.

If too few team leads volunteer, then team leads will take on multiple projects.

If too many team leads volunteer, then the least experienced will just join projects as team members.

It alters the incentive structure a bit: if you want to lead a team, you don't get to vote. The price you pay for power and prestige.

[tannerwelsh]

With above in mind, we'd have two ways to vote:

/vote 2 3
/vote lead

And then the two stages of project formation:

1. Team member assignment: create projects w/ roughly equal ECC between members as close to size as possible
2. Team lead assignment: start with lowest mean ECC team, and find team lead with lowest ECC that is still above the threshold; if not enough team leads, then assign highest-ECC team leads to multiple teams

[tannerwelsh]

From a learner experience perspective, getting to volunteer to be a team lead feels better: I am electing to take on this responsibility, it is not being bestowed upon me through an opaque process.

There's also an element of chance: I can volunteer, but there's no guarantee that I will be chosen for the honor. If I don't have a high ECC, or there are too many volunteers, then I may not get it.

[bundacia]

One wrinkle with that proposed "volunteer leads" tweak is that you'd need a way to go back and put volunteers in teams if too many volunteer.

[bundacia]

I'm having trouble following getting my head around the algorithm. The pseudo-code version originally posted by @tannerwelsh seems to have been tweaked/clarified several times in this thread, and the prototype version @jeffreywescott wrote up is a little to detailed for me to get the overall algorithm out of it. Can we write up the current state of the V1 algorithm again in plain english at a high level (maybe somewhere that is collaboratively editable or in-line comment-able like a PR)?

[shereefb]

I'm a very strong "NO" to the volunteer lead idea. This will be a very difficult thing to change in the learning culture. From the get go, I need learners "surrendering" to echo. We can't give them that agency and then take it away.

@bundacia I'm happy to spend time with you, or who-ever to clarify the algorithm and make it clearer. I can move things around today if that's what's needed.

[jeffreywescott]

@bundacia -- I will try to find time today to write pseudo-code that describes my "prototype" version of the algorithm.

Also, I believe @prattsj is going to start looking at the algorithm this week.

[jeffreywescott]

I have concerns about the volunteer lead idea, also. It seems to me that, for cognitive apprenticeship to work, we need to have a certain number of leads available each cycle.

I recognize that there's a tension between empowering learners to make their own decisions and ensuring that we have enough team leads, but to me, having a guarantee that the cognitive apprenticeship model won't fall apart trumps everything else.

[tannerwelsh]

Thanks for the feedback folks. I like your comment about "This will be a very difficult thing to change in the learning culture." @shereefb. I hear ya, but I'm also gonna keep harping on this point because I think it's worth it.

Being a team lead should be both a duty and a privilege. It should be expected as a rite of passage for all learners, and hold real prestige.

Let me start by telling a story.

As a learner, I see goal voting is an act of exercising agency. I choose the goal that is most relevant & stretchy for me, and I'm excited about it. Then, the results come in, and I'm now team lead for another project. I'm a good player, so I accept the assignment with humility and grace, but I'm not thrilled about it. Because the future that I have been led to expect/hope for (i.e. joining my chosen project) has been replaced with something else as a result of an opaque process that I have no direct influence over, I feel that the LOS is responding less to my actions than to my pre-existing state.

There's nothing inherently wrong with this story, but I think it puts us in danger of turning the team lead position into jury duty. Another service that is both a duty and a privilege, which in theory should be a great honor and celebrated civic service. But, I would argue, the apparent randomness of it makes it feel like a chore. And the systemic effect of that psychological cause is pretty terrible: what should be an act of impassioned leadership becomes a dreaded bureaucratic nightmare that nearly everyone tries to game the system to avoid. The feedback loop from "become a citizen" to "serve on a jury" is just too large and unavailable to people's everyday experience.

I'd rather have a system like student government (pardon the school analogy, it was just the first example to come to mind). People are nominated and/or they volunteer to take on a position of power and greater accountability, and they are rewarded both socially (enhanced status) and academically (extra credit).

We could still require that learners volunteer to lead and incentive that decision through stats allocation; and we could ensure that a minimum ECC delta is met for each team so that cognitive apprenticeship works. A volunteer- or nomination-based approach doesn't intrinsically forfeit any of these options.

Wrapping up this argument, I'll say that I'm ok with starting from a computationally-assigned team lead as a basic, but I'd really like us to move strongly towards a way to add a volunteer/nomination component to set teams up for the best possible leadership experience.

[shereefb]

@tannerwelsh solid point. Perhaps that would be the way to go. We'll probably need to run more experiments to figure it out, and that won't happen before 10 weeks in.

Either way, it's easier to ADD agency than to subtract. So even if we choose to run both experiments, it's way better to start with the computationally-assigned team lead.

[shereefb]

Another improvement that fixes the "I'm yanked off of projects I voted on for jury duty" problem. Let people know that they've been selected for "team lead duty" before they start choosing goals.

[heyheyjp]

@tannerwelsh: I'm not convinced that likening the automatic assignment of team leads (some unknown percentage of the time to projects they might not be interested in) to being selected for jury duty is a fair comparison. I personally think jury duty sucks for a lot of reasons that have nothing to do with the apparent randomness and opaqueness of the selection process. I'd venture a(n admittedly poorly substantiated) guess that there are many people who'd agree. :)

Also, to the point about opaqueness and seeming randomness, I don't know of any reason why the process should be opaque to the learners just because they can't directly influence its outcome. I'd hope that we could be as transparent about it with them as anything else.

I like @shereefb's idea about communicating team lead selection before voting begins. Also finding it hard to argue with the idea that it'd be a lot harder/a much more negative experience for learners to have agency taken away from them than to have it later granted. If we're uncertain about which approach will be more effective, I'd agree that it seems to make more sense to start with more control and to grant more agency to the team members over time.

Other thoughts:

• Learners are going to have an awful lot to take in during the early days as it is. Is there a benefit in "relieving" them of the need to spend mental cycles on whether or not they should volunteer...whether or not they think that those who did volunteer and were selected should be team leads...etc? That's energy/attention that could be spent elsewhere.
• An interesting twist on the idea of granting them agency in the process might be to always have the process be automated but make it possible for cohorts to decide the inclusion/exclusion of and/or relative importance of selection criteria -- the idea being that different cohorts might have slightly different desires for/needs of the people serving on their teams as leads.

[heyheyjp]

@tannerwelsh: also would love to hear more about your thoughts on the merits of a student goverment-like model. My initial gut reaction is that student governments can be highly ineffective and possibly even counterproductive. Systems like these also seem to have a way of favoring the boldest/loudest/most popular. I don't know whether or not that's necessarily a bad thing at this stage, given how carefully we're selecting candidates to begin with. It could be more of an issue at a larger scale? Anyway, it'd be really interesting to continue to explore all of this.

[tannerwelsh]

responding to @shereefb:

Either way, it's easier to ADD agency than to subtract. So even if we choose to run both experiments, it's way better to start with the computationally-assigned team lead.

Yee, I agree.

Let people know that they've been selected for "team lead duty" before they start choosing goals.

Like this too. Maybe before voting learners know that they're "eligible", so they are prompted to (1) not get their hopes up too much for any given project and (2) get excited to possibly lead a team.

[tannerwelsh]

responding to @prattsj

I don't know of any reason why the process should be opaque to the learners just because they can't directly influence its outcome.

Good point. Maybe opaque was too strong a word.

Re: jury duty / student government - I'm not advocating for or against these in their entirety, I was just using them to distinguish between their two separate methods of delegating and selecting people for positions of power. Definitely problems with both beyond just this feature.

Learners are going to have an awful lot to take in during the early days as it is.

Absolutely! Even if we did do volunteering, we wouldn't encourage it early on, because they wouldn't have enough of an experience differential to other learners to actually be selected for a lead position.

the idea being that different cohorts might have slightly different desires for/needs of the people serving on their teams as leads

Love this idea, but it does seem like some v1 (or 2) complexity :) If you can come up with a modularized, customizable selection process that still meets the minimum game criteria and is easily maintained and understood by learners, I'd be thrilled.

Systems like [student government] also seem to have a way of favoring the boldest/loudest/most popular.

True. Again I'm not necessarily advocating for a full adoption of a system like this (in our case, we'd do something like mandatory student government, with a requirement that each student serve at least once or something - the analogy breaks down here), just showing that the emotional/social experience of volunteering and/or being nominated seems to be a more positive one than the "faceless entity draws your name from hat" one, which I worry we'd be at risk of reproducing.

I guess, in a word, I want our team lead selection process to be dignified.

[tannerwelsh]

Thanks for digging into this with me @LearnersGuild/los. Love these discussions.

[tannerwelsh]

One idea for simplifying the criteria for assigning team leads to multiple teams: instead of ensuring that the minimum ECC differential between team leads and the mean of their team's ECC grows linearly with the number of teams a team lead is assigned to, we use the total number of players to being led by a team lead to determine the min ECC differential.

In other words, say for every player a team lead leads, they must have +2.5 ECC above the mean ECC of all players.

Seems like we shouldn't require the same differential when leading a team of 2 (i.e. pairing) as when leading a team of 5 or 6.

So if we have a team with rec'd size 3, with players:

a: 13 ecc
b: 15 ecc

then to lead this team, a player must have ((13 + 15) / 2) + (2.5 * 2), or 19 ECC to lead them.

Here's another example, with a team of rec'd size 6, and two team lead options:

min_diff = 2.5

lead_a = 28
lead_b = 40

players = 10 + 23 + 18 + 17 + 23 = 91
mean = players / 5 = 18.2

min_ecc = mean + (min_diff * 5) = 30.7

diff_a = lead_a - min_ecc = -2.7
diff_b = lead_b - min_ecc = 9.3

In the above case, lead_b option could lead all of the players (regardless of how many teams), but not lead_a.

Just some thoughts for ya @prattsj and @jeffreywescott

[heyheyjp]

@tannerwelsh: that's is an interesting idea.

I'm grappling with a few tensions:

1. I'm currently confused by what appear to be conflicting constraints regarding team lead eligibility and goal disqualification and would love to get some clarity here.

   On the one hand, we have this idea of there being a minimum set of qualifications for assignment as a team lead (whatever those qualifications might finally be settled as). Some ideas proposed that seem to be in conflict:

   • Team leads must have [some amount of] experience greater than the mean of (their teams|the mean ECC of players across all teams they lead?)
   • If a team lead isn't identified, the votes of the players in the team will be excluded (or possibly the entire goal itself? unclear) and they'll be reassigned (really not quite sure i understand this earlier idea correctly anyway, though).
   • Or...team leads don't have to be "fully qualified" team leads at all but could in fact end up being only the most experienced player on an already formed team. In which case it would seem that the point above would never need to be handled? Again, unclear on some things, particularly what are hard constraints and what are ideal scenarios/optimizations.
2. Depending on how the above is resolved, I might have some concern about the potential for us to end up designing a system that is biased towards encouraging work in areas people are most experienced/knowledgeable in rather than the areas (subject to some other reasonable constraints, of course) that people are most interested in or might draw the most benefit from working on.

3. My gut feeling is that the team lead-related requirements as formed so far are way, way too complex for this stage of the program's development--for how little we know about what the learners and the organization will actually need from the team formation process. I'd love to be filled in on what we've determined is the cost of less-than-optimal team assignments and to better understand if/why it's greater than the cost of implementing and iterating on an system this complex out of the gate. Is it worth taking another stab at making potential tradeoffs in...assignment satisfaction?...for the sake of simplicity in implementation and making it easier on ourselves to iterate in the future after we've collected real usage data?

   Can we approach this more simply without sacrificing any truly critical objectives? Experience tells me that that keeping things as simple as possible at this stage is going to be absolutely critical.

   Another idea for an algorithm that isn't entirely different from your initial stab at one but that I think is a bit simpler:

   • Start by assigning every player to their 1st preferred goal (would love to hear the background sometime on why players are limited to 2 votes. it seems somehow at odds with a) the allowance for those votes to be the same value and b) the possibility that they might be assigned to a project for neither goal, which might to some learners seem strange, especially if they might have been placed in another project of interest and weren't only because they weren't able to formally express that interest)
   • Determine the minimum number of leads necessary (sum of (num people in group / rec team size) for all goal groups)
   • Attempt to promote leads to projects within goal groups, attempting to optimize for the fewest possible number of groups (larger group sizes). Rework team sizes if possible to form compliant teams with all of the people in the goal group. Only then, if there are still "leftovers" (a number of people smaller than the minimum recommended team size), reassign those members to a project in another goal.
   • For any projects in a goal-group still left without an eligible lead, make a decision to either pull an eligible lead from another group or to reassign the members of that goal group to projects in other group(s).

   It prioritizes interest in a goal over an attempt to ensure group members are at about the same skill level, which I do understand can be problematic for ecc-based stats in some scenarios. So there's that to be dealt with...maybe.

Please poke lots of holes in it. :)

[jeffreywescott]

Okay, after a deep-dive with @shereefb, followed by an even deeper dive with @prattsj notes here, here's what we came up with:

Plan of Action

Let’s buy more time to do the “phase 2” algorithm right by acknowledging the realities of the “phase 1” algorithm, namely that:

• the only folks who will be team leads in the first 10 weeks are the paid players
• all other team members ​will​ be within 5 ECC of each other, anyway

The first 10 week algorithm could be something fairly simple (full algorithm forthcoming) like just building teams based on 1st choice votes whenever possible, 2nd choice votes if that’s not possible, and randomly otherwise. Then assign one of the team leads* to each team (* team leads determined by pre-computing the players with ecc > N [e.g., N=50 + eccMean of chapter]).

This has the additional advantage of giving us a chance to look at 3-4 cycles of data before we “dig in” to an implementation of an algorithm that takes more things into account.

Other Ideas / Tensions / Questions We're Holding

1. What responsibilities does a team lead have beyond a regular team member? What about privileges? How do we expect team lead behavior to differ from the rest of the team members? Is there guidance we provide?
2. Do we actually ​call​ people “team leads”? If so, will that affect the way people assign relative contribution numbers? Is this okay?
3. It’s important that the project-and-team-formation process ​is​ automated because we don’t want folks to feel like subjective bias can become part of the team formation process.
4. When we revisit the algorithm in week 3 or 4, let’s be mindful about the impact of circular dependencies on the level of complexity required in the implementation and try to avoid them whenever possible. If not, so be it. but if we can, it would seem to be worth it. An example of a circular dependency would be like trying to figure out team leads at the same time as team members based on votes and recommended team size: you don't know who the leads are until you look at the eccMean for the particular team.

[jeffreywescott]

@prattsj -- any chance you could explode my comments above into some kind of pseudo-code for us to comment-on?

The first 10 week algorithm could be something fairly simple (full algorithm forthcoming) like just building teams based on 1st choice votes whenever possible, 2nd choice votes if that’s not possible, and randomly otherwise. Then assign one of the team leads* to each team (* team leads determined by pre-computing the players with ecc > N [e.g., N=50 + eccMean of chapter]).

[heyheyjp]

@jeffreywescott: sure thing - will do

[shereefb]

Team leads should not work on different goals. They could be on multiple teams, but the teams cannot be working on different goals.

[shereefb]

re: calling people team leads. For now, we should track their assignment as leads behind the scenes, but let's not reveal that to players through any UI.

I want to explore how successful we are at getting professional players to just play, instead of "teach", and I'll take every chance I can to make the line blurry between them.

[tannerwelsh]

@jeffreywescott sounds like a good approach. 👍

@prattsj thanks for documenting your thoughts so well. I'm going to assume that you got most of your questions answered yesterday via above-mentioned conversations, but if any are outstanding I can jump in again.

@shereefb I've got a little tension w/r/t not having visible team leads. As a learner, I want to know what the decision-making structure is for a team, what our accountabilities are, etc.

I like starting with prof. players being as close to regular players as possible as an experiment, but once we start having lots of various team leads I think the role deserves a distinguishing title.

[shereefb]

@jeffreywescott +1

It's definitely something on my radar. Again, easier to add a formal role and accountabilities if it turns out they're needed than to remove them. If I had to bet, I would say we're going to need it.

[heyheyjp]

@tannerwelsh @shereefb @jeffreywescott: for your review. Please poke all of the holes and point out any of the gaps. Looking forward to making this better still and finalizing the specs for v0!

---

Team Formation (Super Simple Edition)

Features

Requirements:

• Every player who has voted is assigned to at least one team.
• Players designated as advanced players may be assigned to multiple teams provided that every team they are assigned to is working on the same goal.
  • Players with ECC >= (50 + MEAN ECC) will be designated as advanced players.
• Every team works on exactly one project.
• Every team must include one player designated as an advanced players.

Prioritizes:

• Non-advanced player goal preferences over advanced player goal preferences
• Overall goal popularity over individual goal preference
• Individual preference of players with lower ECC over preference of players with higher ECC (as a tie breaker when all else is equal)
• Slightly higher than recommended team size over slightly lower than recommended team size (when exact recommended team size can't be achieved after meeting higher priority reqs)
• Advanced player capacity over advanced player goal preference. Assumptions:
  • Determining advanced player "capacity" as a function of how many team members they can work with simultaneously is a reasonable and fair thing to do (honestly not sure)
  • Every advanced player will (for the term of this version of the algo) be sufficiently more experienced than every non-advanced player such that they can be considered able to make a meaningful contribution as an advanced player on any team, regardless of the team’s MEAN ECC.
  • Given two sets of teams with different total numbers of non-advanced player and two advanced players with different ECC, the advanced player with the higher ECC is a better fit for the larger player goal group than the advanced player with the lower ECC. (Not sure if this is a reasonable and fair assumption.)

Deferred considerations:

• Nuanced logic regarding advanced-to-non-advanced teammate experience differentials or any relationship between the number of teams an advanced player is assigned to and that advanced player's ECC
  Reason: capacity is known and fixed for the first 10 cycles; very complex for virtually no return on investment in the short term; preferred terms unlikely to remain the same after first data gathered.
• Nuanced logic regarding non-advanced-to-other-non-advanced teammate experience differentials
  Reason: like the above, creates a circular dependency that is expensive to try to manage; risk/cost of less-than-optimal non-advanced player team member equalization appear to be low in the short term; starting ECC is currently identical for all regular members in all chapters and isn't likely to vary greatly between any members over the first 10 cycles)

Other notes:

• Details re: how to obtain recommended team size for a goal can be found here.

Goal Selection & Team Assignment

Overview

• Identify advanced players
• Create goal groups, simultaneously selecting goals and assigning non-advanced players to work on them
• Assign advanced player to goal groups
• Form teams from advanced and non-advanced players assigned to each goal group

Procedure

• Designate advanced players
  • Number of advanced players sets value of MAX SELECTABLE GOALS
• Group all non-advanced players by 1st preferred goal
• Rank goal groups as follows:
  • NUM GROUP PLAYERS (desc)
  • MEAN GROUP ECC (desc)
• While NUM GOAL GROUPS > MAX SELECTABLE GOALS:
  • Reassign players in the lowest-ranked goal group to another group
  • Determine which group each player will be reassigned to with the following order of precedence:
    • Player’s next-most preferred goal
    • Highest-ranked goal group
      Chosen over alternatives A) because other methods that come to mind seem either too complex or based on too-shaky assumptions to be worthwhile at this stage and B) based on the assumption that more players whose preferences can no longer be accommodated would be happier on a more popular goal than being assigned to a less popular goal. Terrible assumption?
  • Repeat Rank goal groups
• While NUM ADV. PLAYERS NOT ASSIGNED TO A GOAL GROUP > 0:
  • Assign unassigned advanced player w/ highest ECC to a goal group in the following order of precedence:
  • Highest-ranked goal group without an advanced player
  • Highest-ranked goal group with an advanced player
• Form project teams. For each goal group:
  • Rank non-advanced players in group by ECC (desc)
  • While NUM PLAYERS NOT ASSIGNED TO TEAM > 0:
  • Specifics TBD. Optimize for max number of groups in following order of precedence:
    • Rec team size
    • Rec team size + 1
    • Rec team size - 1
  • Rank advanced players in goal group by ECC (desc)
  • While NUM ADV. PLAYERS NOT ASSIGNED TO TEAM(s) > 0:
  • Assign advanced player to team in the following order of precedence:
    • Fewest numbers of advanced players already assigned (ideally 0)
    • Greatest number of non-advanced players

[jeffreywescott]

Every player who has voted is assigned to at least one team.

I think all players need to be assigned to teams, unless we're using not having voted as an indication that they have somehow "paused" their game play? @tannerwelsh?

Players may only be designated as a team lead

Does this mean "Players who are team leads cannot also be team members and not team leads"?

LOVE the "Deferred considerations" section. Sounds spot-on for the first version of this algorithm. :thumbsup:

Designate team leads: All players with ECC >= (MEAN ECC + 50)

MEAN ECC of the entire chapter? Wondering if 50 is the right delta here? Hopefully we can make it easily configurable.

Number of team leads sets value of MAX SELECTABLE GOALS

Interesting. If we go this route, I think we should make players very aware of this before they start voting, as it may inspire them to pay more attention to what's already popular rather than selecting something novel.

Assign unassigned team lead w/ highest ECC to a goal group in the following order of precedence:

• Highest-ranked goal group without a team lead
• Highest-ranked goal group with a team lead

I don't understand this -- isn't the number of goal groups == the number of leads? If so, why would we ever need to assign a team lead to a goal group that already has a team lead? I can see how it makes sense for assigning leads to teams, but not goal groups.

[heyheyjp]

@jeffreywescott:

Does this mean "Players who are team leads cannot also be team members and not team leads"?

No...that was just an unintentionally unfinished thought. :) The completed version:

• Players designated as leads may be assigned to multiple teams provided that every team they are assigned to is working on the same goal.
  • Players with ECC >= (50 + MEAN ECC) will be designated as team leads.

MEAN ECC of the entire chapter? Wondering if 50 is the right delta here? Hopefully we can make it easily configurable.

Hm, OK. Yeah, no reason why it shouldn't be easily configurable. Will make it chapter if that's preferable.

Interesting. If we go this route, I think we should make players very aware of this before they start voting, as it may inspire them to pay more attention to what's already popular rather than selecting something novel.

Yes; it's not so much a requirement itself as the implication of another requirement -- that while team leads can work on multiple projects, all projects that they're working on at the same time must be for the same goal. @shereefb made a similar suggestion along with a few others in our quick chat yesterday. I'll write up separate issues to capture those UX ideas.

I don't understand this -- isn't the number of goal groups == the number of leads? If so, why would we ever need to assign a team lead to a goal group that already has a team lead? I can see how it makes sense for assigning leads to teams, but not goal groups.

Not quite. The number of goal groups is constrained by an upper limit equal to the number of team leads. It would be possible for there to be fewer goal groups than there are team leads.

Example: there are 3 team leads, but everyone winds up voting for just 2 goals.

[jeffreywescott]

Thanks for the clarification, @prattsj.

WRT the MEAN ECC being "for chapter", I think that's necessary. As we add more chapters, they won't necessarily be on the same time cycle, so we wouldn't want the ECC of all players, just all players in Oakland (or Brooklyn, or whatever).

Excited to see it come to life in code!

[heyheyjp]

Ahh....I think we were on the same page, then. Apologies - in my head, that was constrained not only by the chapter, but further by only the people who've voted for this cycle in the chapter. Totes aligned now on just chapter, though.

[jeffreywescott]

Ah -- well, in some sense, you're right. We want "active players". If that means only folks who've voted, so be it. Still waiting for @tannerwelsh to respond on that point.

Related: #56 (future)

[shereefb]

I think it's better to assume that players are active if they haven't paused play, instead of assuming they are active if they've voted. In other words, the default should be active and players can opt-out, v.s. the default is inactive and players need to opt-in.

[shereefb]

@prattsj algorithm lgtm for V1. A lot of this will fall apart once we have another cohort in place. We'll probably need to start thinking about clustering players in ECC ranges and determining team leads, and vote eligibility based on these ranges. But that's a lifetime away.

[jeffreywescott]

I think it's better to assume that players are active if they haven't paused play, instead of assuming they are active if they've voted.

We have no way for players to pause play right now.

[shereefb]

Well that makes it easy. We should probably have an active/inactive flag in the DB that we can toggle as admins.

For example, Mihai is only committed to work as a professional player for 4 weeks. After week 4 we need to mark him as inactive if he is not going to work for us.

[tannerwelsh]

Ah -- well, in some sense, you're right. We want "active players". If that means only folks who've voted, so be it. Still waiting for @tannerwelsh to respond on that point.

Agree w/ @shereefb that moderators should have admin abilities to mark players as active/inactive. That way, we always use the set of active player in a chapter to do things like:

• Determine if all players have voted
• Determine whom to assign to teams

[tannerwelsh]

B) based on the assumption that more players whose preferences can no longer be accommodated would be happier on a more popular goal than being assigned to a less popular goal. Terrible assumption?

Not terrible, just hard to conjecture about without data. Fine as a starter.

Pseudocode looks great @prattsj!

[heyheyjp]

@shereefb @jeffreywescott @tannerwelsh: if we introduce a new property active for the player and reference its value when forming projects and teams, should an active user who hasn't submitted any votes be treated according to the rules for someone who couldn't be placed on any teams they did vote for (assigned more or less wherever)...or should the system somehow raise an error and abort while attempting to solicit a vote from anyone who didn't submit one?

[jeffreywescott]

I'd say the former. The algorithm should always produce a result but should warn loudly via logging if the data looks really off somehow (e.g., very few folks have voted).

[shereefb]

Agree with @jeffreywescott I actually think it's going to be pretty common for players to just 'surrender' the system and get whatever project they get assigned to. Wild guess, 1 out 5 people per cycle will choose that road.

[heyheyjp]

K. Thanks, both. I'll put the "warn loudly" bit into another issue for further elaboration as needed.

[heyheyjp]

@shereefb @jeffreywescott @tannerwelsh:

As it is, this latest procedure is such that we could in theory wind up with a situation like this, where the vast majority of players vote for one goal as first preference, and the (very small) set of remaining players wind up in a second goal group. The lead assigned to goal group one would be spread much thinner across teams working on that goal than would the lead assigned to group two.

For this first version, is this acceptable? Or would we want to prioritize a more even lead capacity-to-lead-load balance over accommodating individual player preferences?

Goal 1

L M M M M M M M M M M M M M

Goal 2

L M M