Density, Buoyancy, Buoyancy: Basics 1.2 Postmortem

[samreid]

This issue and issue comment can be used like a journal to jot down ideas, what went well, what could be improved as a resource for discussion at the Buoyancy + Buoyancy: Basics (and Density republication) Postmortem.

• 6/5/2024 SR: In Center and Variability and Buoyancy, each project began with a large legacy codebase which was incomplete and not well-suited for the additional development. Center and Variability began with a prototype which underwent a significant rewrite before we could publish the full version. Buoyancy began with partial and legacy code which was difficult to work with. Having Density with a published phet-io API restricted our ability to make improvements (even with migration rules). As we have discussed at previous postmortems, it is preferable to have as full a design as possible at the beginning, develop all screens/simulations in concert, avoid prototypes, etc, and to work quickly with a high focus to bring it to production efficiently.
• Switching developers and/or designers also incurs a significant cost that we should try to avoid in the future.
• 9/5/2024 SR: Something that worked well in Projectile Data Lab was to work on multiple aspects at the same time. Due to the legacy and history of this simulation, it was more difficult. For future simulations, consider as many of these aspects as possible during initial implementation:
  • test memory leaks
  • keyboard a11y, including but not limited to focus order, grab drag interaction, group sort, hotkeys
  • voicing/description
  • phet-io, including but not limited to state, studio, and reflowing layout when nearby controls are hidden in studio
  • dynamic layout
  • code commenting/documentation and implementation-notes.md, examples.md, and model.md
  • sonification
• 10/3/2024: MK SR AR DL: We wanted to note that the memory footprint of buoyancy was much larger than most sims. It meant that iPhones and iPads could crash very easily. We ended up solving with what felt like workarounds, reducing anit-aliasing computation and simplifying the graphics. If we had know of these problems early, it likely would have resulted in a better solution, like sharing webgl contexts across screens or something. Also noting that this coupled closely with memory leak testing, since even minor memory leaks would result in the sim crashing. Paper trail: https://github.com/phetsims/density-buoyancy-common/issues/316
• SR: Cherry picking to RC branches at the same time as fixing in main using the Maintenance tooling worked great! Lowered complexity, less risky, easy to deploy RC when everything is done. Worked great to have a "commits for rc.2 issue" so we can see everything in order.
• MK and SR found some trouble with the behavior of the reset all keyboard shortcut, which may have ramifications for how we think about global hot keys (testing, knowledge sharing, designing). See https://github.com/phetsims/scenery-phet/issues/874
• Effort Burn Up 2024: https://github.com/orgs/phetsims/projects/88/insights/5
• The ability/authority of the design + development team to decide what to not implement made it possible to make progress. For instance, we were able to decide "that would be too complex and expensive, we should not try to implement it" in a few cases.
• (Will be discussed in PhET-iO Meeting. Do not re-discuss here) SR: Significant amount of time on migration for density at questionable value. 72 nontrivial commits, 647 lines of code. Radical changes to underlying data structures since 1.1 was not well-matched to phet-io. Very brittle. 20+ hours spent on this all told (including 3x devs, consulting with JO, QA testing, addressing CT issues, etc).

[zepumph]

These sims are published now brainstorming attendance list (parentheses are more optional participants): Devs: @AgustinVallejo @samreid @zepumph (@jonathanolson) Designs: @arouinfar @DianaTavares QA: (@Nancy-Salpepi seemed involved in most of the tests.) @kathy-phet

We should try to schedule this for the coming week or two while the project is still fresh in our minds.

[jonathanolson]

each project began with a large legacy codebase which was incomplete

If I recall from the initial development, I had the sim essentially feature-complete to the then-current design doc (some icons and refinements weren't there however), around 4 years ago.

I'm also curious to see how the phet-io instrumentation was (I was the most concerned about that, it was VERY unfinished, and I had no clue how that was going to get completed).

I think it was difficult for me to have significant lengths of downtime not working on it. Initial development felt good (physics libraries made it difficult), but then it seemed like development was paused, and the design was significantly changed across many unanticipated areas, which resuted in a lot of difficulties.

[arouinfar]

Buoyancy suite postmortem

• [x] 1. Schedule the meeting -- October 16, 2024 3:30-5:00 MT

• Invite the entire sim team to participate, and optionally anyone else who had a significant involvement. Have the meeting as soon as possible after the sim is released, so that experiences are still fresh in everyone's mind. Allow sufficient time for the meeting, 1.5-2 hours.

• [x] 2. Enlist a meeting moderator -- @brent-phet

• The moderator's job is to make sure the meeting runs efficiently, stays on point, and focuses on constructive (but not overly negative) criticism. The moderator can be someone not on the sim team, or in many cases the Development Coordinator. The moderator also documents the important points of the meeting.

• [x] 3. Create github postmortem issue and postmortem google doc -- Postmortem doc

• Create a GitHub issue in the repository of the sim for which the postmortem is occurring. This checklist will should be copied into the issue.

• Create a postmortem Google doc for the sim and set phethelp as the owner. This Google doc should live within the appropriate sim folder on Google drive.

• [x] 4. Two successes and two improvements

• Participants will bring a list of no more than 2 items that were done well during the project, and no more than 2 items that could be improved upon. Limiting to 2 requires people to think critically about their experience.

• Consider reflecting on processes, accessibility, novel design components/interactions, and surprises.

• [x] 5. Identifying the top five

• At the meeting, start by having each person present their items (successes, then improvements). Make a list of these items, and note duplicates. Based on popularity, identify the top 5 success items and the top 5 items that need improvement.

• Discuss the 5 success items first. Identify specific things that can be applied to future sims. Celebrate, pat each other on the back. Then move on to the harder part.

• Discuss the 5 improvement items. Try to avoid getting personal. Discuss what was learned, and identify specific things that can be done to prevent these problems in the future. If anything needs more investigation, assign to specific individuals.

• [ ] 6. Action items

• Note important items in the postmortem GitHub issue, and create issues for any tasks the require action, such as updating documents, leftover TODOs, etc.

• Action Item Questions:

  • Suggest updates to project process at next retrospective?
  • Modifications to sim checklist?
  • Messages to share at status?

[brent-phet]

Here are the Action Items:

• Front-load Design

  • MK: But only shifts the burden?
  • -Continue discussion on next sim (Membrane Channels)
  • DL: Design up front before dev kick-off

• Improve Design Process

  • KP: Involve teachers earlier on
  • -Less volume more depth
  • SR: See POSE as a model
  • MK: Make hard choices on feature set
  • -More data for decision-making

• Retrofitting (Legacy Code)

• Introduces mainly complexities

• Will discuss at PhET-iO 10/24 meeting with CM

• All frameworks

• • keyboard support
• • description
• • phet-io
• • dynamic layout
• • query parameters
• • sonification