Issue Dashboard

[samreid]

At today's dev meeting, we agreed we need a dashboard that shows how many issues are assigned to each person, and additionally how many of those are high priority, top priority, ready for review or blocks publication or blocks-sim-publication. @samreid @marlitas and @AgustinVallejo agreed to collaborate on it.

[samreid]

I wrote this script:

// Copyright 2023, University of Colorado Boulder

const buildLocal = require( '../../perennial-alias/js/common/buildLocal' );
const { Octokit } = require( 'octokit' ); // eslint-disable-line require-statement-match
const octokit = new Octokit( {
  auth: buildLocal.phetDevGitHubAccessToken
} );

const getUsers = async () => {
  const result = await octokit.request( 'GET /orgs/phetsims/members', {
    org: 'phetsims',
    per_page: 100
  } );

  // console.log( result.data.length );
  const logins = result.data.map( member => member.login );
  return logins;
};

// https://docs.github.com/en/rest/users/users?apiVersion=2022-11-28#list-users
( async () => {
  const users = await getUsers();

  for ( let i = 0; i < users.length; i++ ) {
    const user = users[ i ];

    const result = await octokit.request( 'GET /search/issues', {
      accept: 'application/vnd.github+json',
      q: `is:issue is:open label:status:ready-for-review assignee:${user} org:phetsims`,
      per_page: 100,
      page: 1
    } );
    // console.log( result.data.items.length );
    // console.log( result.data.items );
    console.log( `${user}: ${result.data.items.length}` );
  }
} )();

Running it showed this issue distribution (ready for review issues only):

~/apache-document-root/main/phet-info$ node dashboard/issueReport.js
(node:39619) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
AgustinVallejo: 1
amanda-phet: 12
apoc64: 0
ariel-phet: 0
arouinfar: 12
Ashton-Morris: 0
BLFiedler: 0
brooklynlash: 0
catherinecarter: 0
chrisklus: 5
CliffordH123: 0
clmccutchan: 0
DevonQui: 0
DianaTavares: 0
emily-phet: 1
issamali67: 0
JacquiHayes: 1
JaronDroder: 0
jbphet: 2
jessegreenberg: 6
jonathanolson: 61
kathy-phet: 10
KatieWoe: 0
kdessau-PhET: 0
Kelly-Bond: 0
liammulh: 0
Luisav1: 6
mariahmaephet: 0
markmorlino: 0
SecondaryRateLimit detected for request GET /search/issues
marlitas: 0
matthew-blackman: 5
Matthew-Moore240: 2
mattpen: 4
megan-phet: 0
Nancy-Salpepi: 0
oliver-phet: 2
phet-dev: 0
phet-steele: 0
pixelzoom: 3
RVieyra: 0
samreid: 1
sarchang: 0
SaurabhTotey: 0
solaolateju: 0
stemilymill: 0
terracoda: 0
veillette: 2
zamb6873: 0
zepumph: 6

[marlitas]

Played around with this tonight to use console.table and create a quick read out of relevant data.

I'm worried about speed on this... It's a long time to wait for all these API calls, and I don't think it's going to be the most used tool if we have to wait that long to get results ( and this is with only getting results for open issues, and ready-for-review). We can keep going this route, but I'm worried it might not be our best option in the long run.

[samreid]

It will be good to discuss in person, so we can cover questions like:

• How often will this script run?
• How fast is fast enough?
• Who will run this script?
• Should it auto-visualize, or output to a format that can be inputted into a spreadsheet? Like stacked bar charts, showing total issues, and stacked by priority (or review)
• Are we visualizing only developers, or all team members?
• Do we need a service like CT where any team member can visualize this?
• How much time is this worth?
• How much will team members need to standardize on semantics of labels in order for this to be useful? (For instance, if some team members use "assigned" to mean they plan to work on it within a month vs some team members use it to mean "I may eventually get to it or maybe not".

Jotting down one idea that may boost the speed significantly (if we decide to go in this direction): Run 40 queries to download all 3888 open issues, and persist them to disk as JSON. This will be a time-consuming step but only needs to be run once. To update it, run 1 query to get 100 issues sorted by "recently changed", and overwrite anything on the disk that has changed. (If more than 100 issues changed, then run successive queries.) Then all issue metadata is offline and we can run any query or visualization quickly.

[marlitas]

It will be good to discuss in person, so we can cover questions like:

100%. And I think these are all great questions that may also need feedback from other debs. I know how I envision the tooling, but that may not align with what everyone else is imagining. We want to make sure that if we put time and effort into this it will be a tool that is actively used by the team to help us be better.

[samreid]

At the dev meeting 5 days ago, @marlitas and @AgustinVallejo and I agreed to work on this issue: https://github.com/phetsims/phet-info/issues/207 which says:

At today’s dev meeting, we agreed we need a dashboard that shows how many issues are assigned to each person, and additionally how many of those are high priority, top priority, ready for review or blocks publication or blocks-sim-publication.

We started with a very productive conversation about the nature of our issues and assigments. To summarize:

• The team discussed the different ways team members interpret issue assignments, and agreed to establish a shared understanding of what it means.
• The team wants to support flexibility in workflow management, but also wants to ensure that issues are addressed and not ignored.
• The team discussed how issue assignments relate to project iterations, and whether or not to unassign everything at the end of each sprint and reassign at the beginning of the next.
• The team also discussed how to handle new issues that are created during an iteration, and agreed that it’s important for team members to communicate and coordinate with each other to solve issues.
• The idea of promoting shared knowledge and promoting productive struggle was also discussed, along with the idea of a shared backlog and assigning effort points to make it easier to add small issues into a sprint.

We thought this was too much to jam in to Thursday’s retrospective, so we wanted to start discussing it now. Full details are in https://docs.google.com/document/d/1bAPEP1iUxQRuCSLHkrwjAH-948kB-sBN3hCC54g-8dc/edit

[marlitas]

@kathy-phet showed me that we can now create workflows in project boards to automatically add issues when criteria are met in repos we want to designate. The https://github.com/orgs/phetsims/projects/67/views/4 board shows how we can really leverage this to be a repo health check including: group by assignee, grouping by repo, burn up charts, etc. I think it might be worth putting our time here...

[marlitas]

^^^ Nevermind... upon further experimenting it limits us to adding up to 4 custom workflows

[kathy-phet]

Others are "runing their own GH Action" to sync their project with multiple repos. So looking at GH Actions for bringing all issues to a project board could be a good way to go. Then you get all that a project board has to offer.

[kathy-phet]

Up vote these 2 Github issues: https://github.com/orgs/community/discussions/47803 https://github.com/orgs/community/discussions/44826

[jessegreenberg]

There is a problem on CT because of the octokit dependency added for this issue.

npm ERR! code EINVALIDPACKAGENAME
npm ERR! Invalid package name "@octokit" of package "@octokit@~4.2.0": name can only contain URL-friendly characters.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Jesse\AppData\Local\npm-cache\_logs\2023-02-23T16_15_53_245Z-debug-0.log

I tried typing npm install --save octokit and it added a dependency like this (notice there is no @)

  "dependencies": {
    "octokit": "^2.0.14"
  }

But it is also at version 2.0.14 and I see ~4.2.0 checked in. I tried to remove the @ from what is checked in and still saw the same error. Then I found specific modules like @octokit/core at https://www.npmjs.com/package/octokit. Changing to @octokit/core makes the error go away but I don't know if it will work for this issue. But I will commit that for now to fix CT.

[samreid]

My subteam is not focusing on this issue, so I'll self-unassign for now.

[samreid]

Based on yesterday's conversation about issue management, I wanted to get a quick look at how many open issues are assigned to each member of the org:

``` AgustinVallejo: 23 Ashton-Morris: 5 BLFiedler: 33 CliffordH123: 0 DevonQui: 0 DianaTavares: 10 JacquiHayes: 8 JaronDroder: 0 KatieWoe: 40 Kelly-Bond: 1 Luisav1: 31 Matthew-Moore240: 11 Nancy-Salpepi: 0 RVieyra: 7 SaurabhTotey: 1 amanda-phet: 82 apoc64: 0 ariel-phet: 2 arouinfar: 71 brooklynlash: 0 catherinecarter: 4 chrisklus: 145 clmccutchan: 2 emily-phet: 17 issamali67: 1 jbphet: 87 jessegreenberg: 224 jonathanolson: 462 kathy-phet: 147 kdessau-PhET: 5 liammulh: 8 mariahmaephet: 2 markmorlino: 3 marlitas: 24 matthew-blackman: 12 mattpen: 185 megan-phet: 1 oliver-phet: 41 phet-brent: 0 phet-dev: 0 phet-steele: 1 pixelzoom: 5 samreid: 3 sarchang: 0 solaolateju: 7 stemilymill: 2 terracoda: 34 veillette: 25 zamb6873: 6 zepumph: 23 ```

[kathy-phet]

@samreid - Is there a way to print an open issues per repo table (similar to above but per repo)?

[samreid]

The process above is not amenable to reporting per repo, it would probably take 30-60 minutes to write a script that uses the GitHub API to make a per repo table.

[kathy-phet]

Hey Sam, If that fits into your issues of choice, I think we would find that script useful as we work on approaches and any sort of sprints on addressing and reducing our open issues.

[samreid]

@AgustinVallejo and I wrote a script that indicated assignees by repo (note that issues can be double counted if assigned to more than 1 person)

``` AgustinVallejo 19 { "chipper": 3, "energy-skate-park": 1, "keplers-laws": 3, "my-solar-system": 8, "phet-info": 3, "special-ops": 1 } Ashton-Morris 5 { "a11y-research": 1, "my-solar-system": 1, "number-line-integers": 1, "paper-land": 1, "quadrilateral": 1 } BLFiedler 31 { "a11y-research": 3, "community": 3, "fourier-making-waves": 1, "my-solar-system": 1, "paper-land": 10, "quadrilateral": 11, "ratio-and-proportion": 1, "tasks": 1 } CliffordH123 0 {} DevonQui 0 {} DianaTavares 11 { "blackbody-spectrum": 1, "buoyancy": 1, "density": 1, "my-solar-system": 2, "special-ops": 1, "tasks": 1, "website": 4 } JacquiHayes 8 { "fourier-making-waves": 1, "phet-io": 3, "phet-io-sim-specific": 1, "phet-io-wrappers": 1, "website": 2 } JaronDroder 0 {} KatieWoe 42 { "area-builder": 1, "build-a-nucleus": 1, "capacitor-lab-basics": 1, "chipper": 1, "circuit-construction-kit-black-box-study": 1, "circuit-construction-kit-common": 1, "curve-fitting": 1, "decaf": 2, "density": 2, "energy-skate-park": 2, "geometric-optics-basics": 1, "john-travoltage": 1, "joist": 1, "number-compare": 1, "number-play": 1, "phet-android-app": 2, "phet-ios-app": 2, "qa": 8, "quake": 1, "scenery": 3, "tasks": 1, "waves-intro": 1, "website": 3, "website-direct": 1, "website-meteor": 2 } Kelly-Bond 1 { "phet-ios-app": 1 } Luisav1 31 { "build-a-nucleus": 30, "query-string-machine": 1 } Matthew-Moore240 9 { "a11y-research": 2, "molarity": 5, "paper-land": 2 } Nancy-Salpepi 2 { "website-direct": 2 } RVieyra 7 { "VirtualWorkshop": 2, "tasks": 1, "website": 3, "website-meteor": 1 } SaurabhTotey 1 { "tambo": 1 } amanda-phet 82 { "area-model-common": 4, "area-model-decimals": 1, "arithmetic": 1, "binder": 2, "calculus-grapher": 1, "center-and-variability": 7, "curve-fitting": 1, "eating-exercise-and-energy": 3, "equality-explorer": 8, "equality-explorer-basics": 1, "equality-explorer-two-variables": 1, "expression-exchange": 1, "fractions-common": 1, "fractions-equality": 1, "fractions-mixed-numbers": 1, "function-builder": 1, "graphing-quadratics": 2, "least-squares-regression": 3, "make-a-ten": 1, "mean-share-and-balance": 8, "natural-selection": 3, "number-compare": 1, "number-line-distance": 3, "number-line-integers": 2, "number-line-operations": 1, "phet-info": 2, "plinko-probability": 1, "scenery-phet": 3, "sun": 1, "tasks": 6, "vegas": 1, "website": 4, "website-meteor": 5 } apoc64 0 {} ariel-phet 2 { "build-a-nucleus": 1, "tasks": 1 } arouinfar 70 { "acid-base-solutions": 1, "build-a-molecule": 2, "build-an-atom": 1, "buoyancy": 1, "capacitor-lab-basics": 1, "charges-and-fields": 2, "chipper": 1, "collision-lab": 3, "density": 1, "energy-forms-and-changes": 2, "energy-skate-park": 3, "forces-and-motion-basics": 2, "fourier-making-waves": 2, "gas-properties": 1, "geometric-optics": 2, "geometric-optics-basics": 1, "gravity-and-orbits": 1, "greenhouse-effect": 4, "masses-and-springs-basics": 1, "models-of-the-hydrogen-atom": 8, "molecule-polarity": 1, "molecule-shapes": 1, "phet-info": 2, "phet-io": 1, "projectile-motion": 9, "scenery": 1, "scenery-phet": 2, "studio": 1, "sun": 2, "vector-addition": 1, "vegas": 1, "wave-interference": 2, "wave-on-a-string": 5, "website": 1 } brooklynlash 0 {} catherinecarter 4 { "data-fluency": 1, "equality-explorer": 3 } chrisklus 142 { "aqua": 2, "axon": 1, "balancing-act": 1, "binder": 1, "blackbody-spectrum": 5, "build-a-nucleus": 2, "build-an-atom": 1, "center-and-variability": 16, "chipper": 5, "color-vision": 1, "counting-common": 1, "energy-forms-and-changes": 14, "joist": 3, "make-a-ten": 6, "masses-and-springs": 4, "masses-and-springs-basics": 1, "number-compare": 11, "number-play": 3, "number-suite-common": 9, "perennial": 2, "phet-core": 1, "phet-info": 2, "phet-io-test-sim": 5, "phet-io-website": 1, "phet-io-wrapper-arithmetic": 1, "phet-io-wrapper-lab-book": 1, "phetmarks": 2, "plinko-probability": 1, "query-string-machine": 1, "scenery-phet": 1, "shred": 1, "sun": 6, "tasks": 3, "utterance-queue": 2, "vegas": 1, "website": 3, "website-meteor": 19, "website-scenery": 2 } clmccutchan 2 { "tasks": 2 } emily-phet 17 { "a11y-research": 5, "joist": 1, "paper-land": 3, "ratio-and-proportion": 2, "rosetta": 1, "sun": 1, "tambo": 3, "website": 1 } issamali67 1 { "build-a-molecule": 1 } jbphet 87 { "a11y-research": 1, "area-builder": 2, "arithmetic": 2, "balancing-act": 1, "build-a-molecule": 1, "build-an-atom": 2, "chipper": 7, "color-vision": 2, "equality-explorer": 1, "expression-exchange": 1, "friction": 1, "greenhouse-effect": 13, "number-line-distance": 1, "paper-land": 8, "phet-info": 1, "phetcommon": 1, "quadrilateral": 1, "quake": 2, "rosetta": 4, "scenery-phet": 3, "shred": 2, "states-of-matter": 3, "sun": 4, "tambo": 20, "tappi": 2, "vegas": 1 } jessegreenberg 222 { "a11y-research": 15, "aqua": 1, "area-model-common": 1, "area-model-introduction": 1, "balloons-and-static-electricity": 4, "build-an-atom": 2, "chipper": 6, "circuit-construction-kit-common": 1, "energy-skate-park": 5, "energy-skate-park-basics": 3, "example-sim": 1, "faradays-law": 3, "forces-and-motion-basics": 8, "gravity-force-lab-basics": 1, "greenhouse-effect": 1, "inverse-square-law-common": 1, "john-travoltage": 3, "joist": 10, "kite": 1, "mean-share-and-balance": 1, "molecules-and-light": 5, "ohms-law": 1, "paper-land": 4, "perennial": 5, "phet-core": 1, "phet-info": 1, "phet-io-wrapper-classroom-activity": 1, "phet-io-wrapper-haptics": 1, "plinko-probability": 1, "quadrilateral": 26, "rutherford-scattering": 2, "scenery": 48, "scenery-phet": 16, "shred": 1, "sun": 25, "tambo": 1, "tappi": 3, "twixt": 1, "utterance-queue": 8, "vegas": 2 } jonathanolson 463 { "aqua": 9, "area-model-common": 5, "area-model-introduction": 1, "axon": 3, "balancing-act": 1, "bamboo": 4, "bending-light": 1, "build-a-fraction": 2, "build-a-molecule": 10, "build-a-nucleus": 1, "bumper": 1, "buoyancy": 20, "capacitor-lab-basics": 3, "chains": 3, "charges-and-fields": 11, "chipper": 45, "collision-lab": 1, "curve-fitting": 3, "decaf": 1, "density": 7, "density-buoyancy-common": 3, "dot": 6, "energy-forms-and-changes": 1, "energy-skate-park-basics": 1, "fluid-pressure-and-flow": 1, "fraction-matcher": 1, "fractions-common": 3, "fractions-equality": 1, "fractions-intro": 2, "fractions-mixed-numbers": 2, "gene-expression-essentials": 1, "john-travoltage": 1, "joist": 15, "kite": 14, "least-squares-regression": 3, "make-a-ten": 4, "masses-and-springs": 1, "mobius": 1, "my-solar-system": 3, "nitroglycerin": 1, "pendulum-lab": 10, "perennial": 24, "phet-android-app": 1, "phet-core": 12, "phet-info": 3, "phet-io": 1, "phetcommon": 4, "phetmarks": 4, "phettest": 1, "query-string-machine": 1, "scenery": 129, "scenery-phet": 10, "smithers": 2, "special-ops": 3, "sun": 19, "tasks": 4, "twixt": 7, "under-pressure": 1, "utterance-queue": 1, "vegas": 2, "wave-interference": 1, "wave-on-a-string": 2, "website": 3, "website-meteor": 1, "yotta": 20 } kathy-phet 148 { "a11y-research": 1, "aqua": 1, "balancing-chemical-equations": 1, "capacitor-lab-basics": 1, "center-and-variability": 1, "chipper": 5, "decaf": 8, "density-buoyancy-common": 1, "equality-explorer": 3, "gas-properties": 1, "graphing-quadratics": 1, "joist": 3, "mean-share-and-balance": 4, "molecule-polarity": 1, "natural-selection": 2, "number-line-distance": 1, "phet-android-app": 3, "phet-info": 2, "phet-io": 7, "phet-io-sim-specific": 1, "phet-io-website": 4, "phet-io-wrapper-arithmetic": 1, "phet-io-wrapper-lab-book": 1, "phet-io-wrappers": 2, "phet-ios-app": 4, "phetcommon": 1, "projectile-motion": 1, "qa": 5, "query-string-machine": 1, "reactants-products-and-leftovers": 1, "rosetta": 1, "scenery": 10, "scenery-phet": 18, "sound": 1, "special-ops": 10, "tambo": 1, "tasks": 5, "vegas": 1, "waves-intro": 2, "website": 19, "website-meteor": 1, "weddell": 1, "yotta": 9 } kdessau-PhET 5 { "special-ops": 2, "tasks": 1, "website": 2 } liammulh 8 { "community": 1, "forces-and-motion-basics": 1, "phet-info": 1, "rosetta": 5 } mariahmaephet 2 { "tasks": 2 } markmorlino 3 { "special-ops": 2, "website": 1 } marlitas 25 { "build-a-nucleus": 3, "number-compare": 1, "number-suite-common": 1, "phet-info": 1, "phet-io-wrappers": 1, "scenery": 12, "shred": 1, "sun": 4, "tasks": 1 } matthew-blackman 12 { "beers-law-lab": 1, "circuit-construction-kit-common": 2, "data-fluency": 2, "phet-io": 2, "projectile-motion": 4, "tasks": 1 } mattpen 185 { "aqua": 1, "chipper": 3, "decaf": 2, "perennial": 13, "phet-android-app": 9, "phet-info": 1, "phet-io-website": 2, "rosetta": 4, "smithers": 2, "special-ops": 4, "website": 74, "website-direct": 24, "website-meteor": 46 } megan-phet 1 { "tasks": 1 } oliver-phet 39 { "a11y-research": 1, "build-a-molecule": 1, "circuit-construction-kit-common": 1, "phet-android-app": 4, "phet-info": 1, "phet-ios-app": 8, "phetcommon": 1, "qa": 1, "special-ops": 1, "tasks": 1, "website": 14, "website-direct": 2, "website-meteor": 2, "weddell": 1 } phet-brent 0 {} phet-dev 0 {} phet-steele 1 { "energy-forms-and-changes": 1 } pixelzoom 9 { "calculus-grapher": 3, "dot": 1, "number-play": 1, "reactants-products-and-leftovers": 3, "scenery-phet": 1 } samreid 4 { "axon": 1, "joist": 1, "phet-info": 1, "scenery": 1 } sarchang 0 {} solaolateju 7 { "my-solar-system": 1, "rosetta": 1, "website": 4, "website-meteor": 1 } stemilymill 2 { "phet-android-app": 1, "website": 1 } terracoda 34 { "a11y-research": 8, "area-model-common": 1, "balloons-and-static-electricity": 1, "binder": 1, "chipper": 1, "gravity-force-lab": 1, "gravity-force-lab-basics": 1, "john-travoltage": 1, "joist": 1, "paper-land": 2, "phet-info": 1, "quadrilateral": 4, "scenery-phet": 2, "sun": 1, "tambo": 1, "tasks": 3, "website-meteor": 4 } veillette 25 { "calculus-grapher": 5, "charges-and-fields": 3, "dot": 4, "optics-lab": 11, "sugar-and-salt-solutions": 2 } zamb6873 6 { "VirtualWorkshop": 5, "website": 1 } zepumph 21 { "aqua": 1, "axon": 2, "chipper": 4, "my-solar-system": 2, "phet-info": 1, "phet-io": 2, "phet-io-sim-specific": 1, "phet-io-wrappers": 1, "scenery": 2, "sun": 2, "tandem": 2, "tasks": 1 } ```

[samreid]

More importantly, we used the octokit API to cache all open issue metadata locally (around 20MB), so it will be easy and fast to run other reports.

[samreid]

Some amazing progress in visualizing issues over in https://github.com/phetsims/phet-io/issues/1914#issuecomment-1483313422

It made it easy to view issues across repos and see how they are assigned, labeled, etc.

We considered a GitHub action to put all our common code issues on one project board, so we can use "insights" to visualize it all.

Update: April 18, 2022 - Elaborating on the proposal

• Identify common code repos. This would include things like scenery, tambo, shred, phet-io, etc but not ones that are too sim-specific like density-buoyancy-common or circuit-construction-kit-common, or side projects like phet-io-website. You can see the beginnings of this list in build.json
• Delete other Project Boards
  • PhET-iO Backlog
  • Ready for Review Triage Project
  • Common Code Issue Dashboard
  • Review the 17 total project boards and see which can be deleted: https://github.com/orgs/phetsims/projects?query=is%3Aopen
• Create a new Project board called "Common Code" or come up with a better name
  • Initially populate it with open issues in common repos. Could be around 1000 issues. (Note there is a 2500 cards per column limit, so we would be safe for a while: https://docs.github.com/en/enterprise-server@3.4/issues/organizing-your-work-with-project-boards/tracking-work-with-project-boards/adding-issues-and-pull-requests-to-a-project-board )
  • Add GitHub Actions so that issues opened in these repos automatically appear in the project board
  • Add an Automation that closed issues are moved to the Done column.
  • This project board can be used to simulate the "ready-for-review" or "phet-io" project boards by label filtering.
• Ongoing/maintenance costs
  • When a common code repo is added or deleted, we would need to update the GitHub Actions
  • When the "Done" column fills up, rename it to "Done - Archive 1" and create a new "Done" column. Update the automation to point to the new "Done" column.

UPDATE: I saw a report that said a project board was capped at 1200 items. https://github.com/orgs/community/discussions/9678 That seems at odds with the 2500 cards/column note mentioned earlier.

UPDATE: I also see there are some automations that don't need Actions, https://github.com/orgs/phetsims/projects/68/workflows