PhET-iO instrumentation sprint

[pixelzoom]

@kathy-phet and @marlitas asked for an estimate to instrument this sim, including the Game screen. To inform that estimate, I'm going to:

• Review @LindaStegemann's document, PhET-iO Game Design Number Pairs / Balancing Chemical Equations, which contains user stories for this sim.
• Do a PhET-iO instrumentation sprint, to identify what's easy, what's difficult, what requires more info, etc.

While I'm in the code, I'm going to instrument anything that's straightforward, because it will be more cost-effective to just do it while I'm there than to come back later. Having a partial Studio tree may also give designers some insight into how the Game is implemented.

[pixelzoom]

Some straightforward elements were instrumented in the above commits, along with a minor bit of separating and renaming elements.

As a result of this sprint, I have a lot of PhET-iO design questions - see below. It's impossible to provide an estimate without discussing these, so I'll schedule a "design sprint" with @arouinfar.

---

BCE PhET-iO design questions

Intro screen

1. Should the view for each equation be treated as a separate scenes, or as 1 scene? For example, do I want to hide the Tools combo box for all equations, or just for 1 equation? Independent equations with their own pickers. Everything else global.

2. Should accordion boxes be hideable? ✅ Yes.

3. Should individual pickers be hidable? Per equation or globally? Yes. Per equation.

4. Should stacks of atoms be hideable? ✅ No.

5. Should parts of a “balanced representation” (balance scales, bar charts) be hideable? Per equation or globally? ✅ No, hide the Tools combo box. Might be nice to have Properties in the model that tell you how many of each atom you have.

6. Should the smiley face be hideable? ✅ Do not instrument.

7. Should anything about the behavior when the equation is balanced be customizable? It current shows a smiley face, makes the arrows yellow, change the != sign to a yellow = sign. ✅ No.

8. Should a notification occur when the equations is balanced? Unbalanced? isBalancedProperty in the model.

9. Should the blue bar at the bottom of the screen be customizable in some way? If you hide it, you can’t see the white labels on the radio buttons. If you hide the radio buttons, should it also automatically be hidden? ✅ Hide the bar when radio buttons are hidden.

Game screen

10. Should levels be treated like scenes, having their own state? Yes.

11. Should the UI for each level be configrable independently or globally? For example, if I want to hide something, should I have to do so in each level? Or should hiding something for one level hide it for all levels? There may not be a single answer — it may depend on what element we’re talking about. Most flexible is for each level to have it’s own UI, and customize that UI.

12. Games typically operate like a state machine - see gameScreen.model.stateProperty. Is this what you expected? Is this appropriate for PhET-iO clients? ✅ This seems OK not that confusing. Probably wouldn’t feature.

13. Should each level have its own state machine, or is it sufficient to have 1 state machine for the Game screen? ✅ One state machine for the Game screen.

14. What should be part of the state for each level versus the state for the Game screen? challengeProperty, pointsProperty, … Unsure. It might be OK to have things like challengeProperty and pointsProperty at global level, because saving their state per level is not relevant.

15. What would you expect setting gameScreen.model.levelProperty to do? Change the view to the level that is set. So we need levelProperty.value = null when on the level-selection UI.

16. Do we need to instrument the Game UI? Like other screens, or less? Is correct operation in the State Wrapper, or record/playback, important? We should try to instrument the UI like other screens as part of MVP.

17. Do we need to be able to hide the level-selection UI altogether and make the Game screen a single level? ✅ Don't worry about this for now.

18. Instead of Properties like numberOfEquationsProperty and currentEquationProperty, should we use “challenge” terminology? E.g. numberOfChallengesProperty, currentChallengeProperty. Yes.

19. Should any of these constants be configurable via PhET-iO? Per level or globally? Any others?

    • const EQUATIONS_PER_GAME = 5
    • const POINTS_FIRST_ATTEMPT = 2
    • const POINTS_SECOND_ATTEMPT = 1 _EQUATIONS_PERGAME - We would need to define a safe range that is guaranteed to work with the challenge generator. And it has to be a query parameter or we can’t configure things like the scoreboard on startup. _POINTS_FIRST_ATTEMPT, POINTS_SECONDATTEMPT - We could do these fairly easily.

20. Should the number of attempts allowed be configurable, or even have no limit? How is does this work with points awarded? Allow 1 or 2 via a Property and/or query parameter. Allowing > 2 or no limit is big problems.

21. Should the “balanced representation” used for each level be configurable? It is currently:

    • level 1 = balance scales
    • level 2 = balance scales or bar charts, randomly chosen per challenge
    • level 3 = bar charts Add a Property per level, with values ‘balancedScales’, ‘barCharts’, ‘bothRandom’ for customizing. Not terribly difficult, but adds a test burden.

22. Should best scores and best times be stateful? Yes, client should be mindful not to unintentionally prepopulate these by playing the game in Studio.

23. Should the timer’s elapsed time be stateful? Yes.

24. When a challenge correctly (or incorrectly), how should PhET-iO be notify? Emitter? What parameters to the Emitter? (challenge info, score, number of attempts, time to solve,…) Without anything special for PhET-iO: Watch for checkButton press, or observe `isBalancedProperty. When it goes false -> true, then interrogate other Properties as needed. With special PhET-iO code: Emitter when the check button is pressed, with parameter for isBalanced and other state. Emitter when level is completed. These will need to be designed and add a test burden. Look at what I did in Fourier.

25. In some games, the student can continue to play with a challenge after solving it. What notification happens in this case? ✅ Not relevant in this game, but generally responsibility of the wrapper.

26. Games are different than other screens in that they (sometimes) have a timer. Should we keep track of time to solve a challenge regardless of whether it’s selecting in the UI, and always include it in PhET-iO notification? Or should the client wrapper be responsible for tracking time? Yes, include elapse time in state.

27. Same question for best score. Yes, include best score in state.

28. Should the order of levels be configurable? Not currently supported by ?gameLevels. ✅ No.

29. Do we need to be able to skip the “Game Results” dialog? Or the game reward? ✅ For now, no. Too difficult without concrete client requirements.

30. What should be customizable about the level-selection buttons? For each button individually, or globally? Button visibility. Text label visibility. Hide stars? This require common-code changes in vegas.

31. What should be customizable about the Scoreboard? (top bar in each level) For each level indivisually, or globally? Visibility of each of the elements This require common-code changes in vegas.

32. When should the Check button be enabled? When you’ve changed one of the spinner values? When all spinners have a non-zero value? Is this requirement configurable? Wrapper around checkButton.visibleProperty so that clients can control its visibilty. Otherwise leave the sim behavior as is.

[pixelzoom]

@arouinfar and I reviewed the design questions in https://github.com/phetsims/balancing-chemical-equations/issues/160#issuecomment-2395557012. Our (tentative?) design decisons were added in italics.

[LindaStegemann]

Unless a number has a comment below, I fully agree with the information stated above:

4. Clarification - we're saying "if the accordion is visible, the user will not be able to make the atoms not visible?" If that statement's answer is "yes", I'm in agreement. The expectation would be that the white box is the atoms box - if you don't want to show atoms, hide the whole box. A solid white box without change for the student would be confusing. CM: Correct. To clarify further, the contents of the accordion box will not be customization. The accordion box itself will be customizable in the ususal ways.

5. As the studio designer, if I want them to have the bars tool (as an example), I would just set the state, then hide the dropdown, giving them bars without the risk of them removing it. Would this be doable under the MVP? CM: Correct.

6. As I view the sim, the reset is on that ribbon. If you hide the radio ribbon, what happens to the reset button? Would you be able to have the reset button without the ribbon? CM: Visibility of the bar matches the visibility of the radio buttons. The resetAllButton is independent.

7. Each level - that said, when this goes to PhET-iO for teachers, this may need to be something considered in the move. While I think PhET-iO partners will not see that as unusual, teacher users will likely notice and wonder about the "no universal setting" thing. CM: Configuring per level provides the most flexibility and easiset to implement, and it's consistent with PhET's approach to instrumenting "scenes", so we recommended this approach. Configuring only globally makes it impossible to configure per level and introduces PhET-iO-only elements that make the implementation more difficult and increase the QA burden. Providing both global and per-level customization is imo prohibitively expensive.

8. _"EQUATIONS_PERGAME - We would need to define a safe range that is guaranteed to work with the challenge generator. And it has to be a query parameter or we can’t configure things like the scoreboard on startup." Food for thought: I suspect the ranges the PhET-iO designers will want to use are proportional to the max. So, if there are 20 equations available in Level 1, and the default is 5 equations in Lv 1, I see folks adjusting it to 10 (half the max) and 20 (the max) most frequently. When this goes into the teacher version, I see them specifically adjusting to increments of 5, 10, and 25 for grading. (We like "10" for quizzes - makes the math easier.) CM: What we called the "safe range" for number of challenges will vary by sim. In BCE, Level 3 has complicated rules about what challenges must subsequently be exclused when a challenge is generated. So choosing 1 challenge from the pool may shrink the pool by > 1 challenge. If we run out of challenges, we're in trouble.

9. For the sake of argument, let's say I (a PhET-iO user) didn't read the "Read Me" file before I got started. If I did play these in studio, would I be able to fix my mistake in Studio by resetting a field or deleting a value? CM: Unlikely. Properties that are set by the sim are typically marked as "ready only" in the PhET-iO API. So you would not be able to (for example) delete the elapse time or scores. With great power comes great responsibility, so read the manual :)

10. Games are different than other screens in that they (sometimes) have a timer. Should we keep track of time to solve a challenge regardless of whether it’s selecting in the UI, and always include it in PhET-iO notification? Or should the client wrapper be responsible for tracking time? Ideally, we would track it regardless of if it's turned on. There's a definite use case for the student to not know they're being timed, but for the instructor to receive time data. (Teaching x Anthropology lessons coming back: "the less the subject knows they're being observed, the more normal they will act, and thus the better observations you will get"...) CM: For Games that have a timer, it's low overhead for us to always keep track of the time internally and provide it in the PhET-iO API. For Games that do not have a timer, there will need to be discussion. For non-game screens, it is the responsibility of the client's wrapper to track time between "events".

11. I prefer "hide stars" because the "stars =?= grade" questions gets old. This may also be important if the client has their own grading scheme based on values coming from the game - they may give the student a 75% but the student had 3/5 stars... CM: Noted. We put a "?" after "Hide Stars" because this will require common-code changes, and is therefore significantly more expensive.

[pixelzoom]

Thanks for taking the time to review @LindaStegemann! I've added inline responses to your comments/questions in https://github.com/phetsims/balancing-chemical-equations/issues/160#issuecomment-2399724799. See CM:...

I should also note that we did not address several of the user stories in @LindaStegemann's document because they are difficult. We're anticipating that we'll discuss those in another TBD forum, to decide if they are needed for the MVP.

[pixelzoom]

At 10/10/24 iteration planning meeting, this work was put on hold.