Add customizations to the keyboard help dialog

[samreid]

In https://github.com/phetsims/center-and-variability/issues/257#issuecomment-1604609188 @emily-phet said:

I believe this sim may benefit from having some customizations in the keyboard help dialog. For example, I think it currently has information re sliders (which is great!) but nothing looks like a slider. It would be nice to have language added that indicated predict and interval tool. It could also be helpful to have information indicated picking up and moving objects (soccer balls and cards).

@catherinecarter can you please review a few other sims with a keyboard help dialog and recommend how to proceed?

Also I should mention that it has been straightforward in the code to implement things using the AccessibleSlider interface, which sounds appropriate based on the documentation at the top of the AccessibleSlider *.ts file.

This could be used by anything that moves along a 1-D line

[catherinecarter]

I've created a mockup for all three screens. @emily-phet helped with understanding what needed to be included. I don't think I missed anything, but anyone looking, please let me know if I did. One question I had is about the "move in smaller steps" idea. I didn't identify in the text what was being moved in smaller increments, but thought there's a chance the user will know the description would be referring to the thing directly above it. So on the variability screen, for example, the interval tool handle is listed, and directly below it is "Adjust in smaller steps." Then, the interval itself (for translating) is described, and then "Move in smaller steps" is directly below. Is this how it would typically be done? All the examples I references had only one thing to adjust or move in smaller steps, so it was very clear in those sims.

Median screen:

Mean & Median screen:

Variability screen:

[emily-phet]

I think this looks great! If you wanted to streamline the Variability screen dialog, you might go with "Move interval tool handle", "Move Interval", "Move in smaller steps" to consolidate a bit. Up to you!

[catherinecarter]

Yes, that sounds good, @emily-phet, thanks. So on the Variability screen, the left hand side will read:

[zepumph]

I spoke with @catherinecarter and @marlitas today during our standup meeting about the customizability of the generic slider section. It isn't clear if "Objects" is clear enough, but we could change that word per screen if needed. Here is a mockup and the patch to get us there. I also committed a couple of updates to the demos to show this behavior better.

```diff Subject: [PATCH] update demos with more content, --- Index: js/common/CAVScreen.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/common/CAVScreen.ts b/js/common/CAVScreen.ts --- a/js/common/CAVScreen.ts (revision 4f3c5b44e56655b27e5fd0501af8765aea4be98f) +++ b/js/common/CAVScreen.ts (date 1688659918799) @@ -15,6 +15,7 @@ import PickRequired from '../../../phet-core/js/types/PickRequired.js'; import SliderControlsAndBasicActionsKeyboardHelpContent from '../../../scenery-phet/js/keyboard/help/SliderControlsAndBasicActionsKeyboardHelpContent.js'; import CAVModel from './model/CAVModel.js'; +import TinyProperty from '../../../axon/js/TinyProperty.js'; type SelfOptions = EmptySelfOptions; export type CAVScreenOptions = SelfOptions & PickRequired; @@ -25,7 +26,13 @@ const options = optionize()( { backgroundColorProperty: CAVColors.screenBackgroundColorProperty, - createKeyboardHelpNode: () => new SliderControlsAndBasicActionsKeyboardHelpContent() + createKeyboardHelpNode: () => new SliderControlsAndBasicActionsKeyboardHelpContent( { + sliderSectionOptions: { + verbStringProperty: new TinyProperty( 'Move' ), + headingStringProperty: new TinyProperty( 'Move Objects:
soccer balls, interval tool, and others' ), + sliderStringProperty: new TinyProperty( 'objects' ) + } + } ) }, providedOptions ); super( createModel, createView, options );

[emily-phet]

I'm guessing it was concluded that custom structure here was not ideal.

I do think this solution is not very clear. For example, soccer balls and the interval tools are very different things and not all of the controls map to all of the objects. For example, you can't move the soccer balls in small steps.

From a learner perspective, lumping all of these very different things together under "objects" seems likely more confusing than helpful in this context.

[emily-phet]

Maybe the problem here is treating all of these as if they are "sliders". They should not be. If the soccer balls are sliders under the hood...that's not ideal. Remember, some users will be able to access information about the component type, and it would be very odd to try to interpret the behavior in the sim of the soccer balls as a slider.

[zepumph]

Hmmm. I wonder how we make a soccerball feel less slider-y while still utilizing the ideal "1-d dragging" common code that AccessibleValueHandler is there for. Can we add an aria-roledescription to override its "slideryness"? Also perhaps this is OK for alt-input so long as we come up with a better solution come time for full description.

What about using multiple copies of the Slider controls for clarity. One says "Move Soccer Balls" and the other says "Adjust interval tool" and the third says "Change remaining objects" (not actual wording, but you get the idea). Nothing is saying that we need to be sparse.

[marlitas]

If the soccer balls are sliders under the hood...that's not ideal.

This concerns me because using AccessibleSlider for components that are not "sliders" is a common practice in our codebase.

Here are some examples of objects that extend AccessibleSlider that are not sliders:

• John Travoltage: AppendageNode
• Gravity Force Lab: MassNode
• CoulombsLaw: ChargeNode
• Ratio and Proportion: RatioHandNode

They all have slider type behavior which is why extending AccessibleSlider makes sense, but the implementation detail would be confusing to a user when comparing these objects to an HSlider for example. In addition, as a new developer on the team my understanding is that it is encouraged to extend AccessibleSlider for "1-d dragging" components no matter how "slidery" their representation is. If that is not the case I think this needs to be communicated to the dev team in general.

I am also going to mark this as a high-priority issue because it will be a large refactor to have SoccerBall not extend AccessibleSlider and implement the "1-d dragging" separately especially with alt-input support. We should come to a decision quickly on what implementation strategy we want to use here.

[emily-phet]

@marlitas the use of AccessibleSlider for interactions that do not have the traditional appearance of sliders I agree is common practice, one that arose from work across over a dozen sims by the inclusive design team, originating back in 2015-2016. I offer this information as you mention you are a newer developer, so perhaps there is history here you haven't encountered yet. I'd be happy to share more about that in a synchronous discussion if that would be helpful.

For the full team - There's a little bit of additional nuance to that, relevant here to the soccer balls. When thinking about a wide range of users and contexts, the decision about what components to use for the desired interaction is most robust when it's taken into consideration that:

• some users will only know what the object appears to be (in the case in question here, soccer balls),
• while other people will have the experience of both the soccer appearance and component type (in this case, soccer balls and sliders),
• while others will only have the component type and name/label (presumably something like "soccer ball X slider"). From this perspective, I'm not sure how a learner that knows the component type is a slider would interpret the behavior and engage in sense making with the soccer balls being sliders - that feels like a difficult mapping to me. Confounding factors include the fact there can be so many of them, they appear and disappear, and there is also a visual representation of "stacking". I'm not sure how someone would make sense of that when introduced to the soccer balls as sliders. So, therein lies my concern.

Given that it's probably long since was decided the soccer balls should be sliders, and accepting that, for the situation of the keyboard help dialog, it's not clear to me why the original mockups @catherinecarter created cannot be implemented. There is a lot of subtle, nuanced, effort in the organization and word selection incorporated around words like "move" (which are associated with objects less like numerical sliders) and "adjust" for objects more like numerical sliders, etc., to help cue the learner not only how to make use of each of these, but also productive ways to think about these interactions. The existence of standard patterns in the help dialog are not intended to be shoehorned (so to speak) into sims. Rather, some sims do have standard components that appear as standard components and so those patterns can be used as is, while other sims have other situations, including standard components that appear otherwise where more language is necessary to help make the dialog understandable (as in this case), or less standard interactions that require custom additions in the keyboard help dialog.

Can someone clarify why it was decided not to implement @catherinecarter's original mockups? Is there a specific barrier, or was it determined that simplifying the difference from pre-existing structures would be more efficient? Or something else?

[catherinecarter]

@emily-phet, thanks for your input and clarity of the history of the use of the accessibleSlider. It's very valuable for both myself and others who are newer to project to understand the reason for decisions, as well as opportunities to learn more about consistency within the project.

Based on yours and others' feedback, I feel there are two things to think about here, one being the design and the other being the implementation.

In terms of the design, when @marlitas, @zepumph, and I met to discuss the keyboard dialog last week, I did not fully understand what the common code actually was. Last week, I learned the noun and the verb are customizable in both the title and the lines below the title, and therefore thought that meant for an infinite number of lines to describe what objects are moveable (like, "Move soccer balls" on one line, and "Move predict median slider" on another line).

Today while meeting, I gained clarity. I now know the common code includes the entire block of text rather than line-by-line customization of the noun and the verb. I learned if one of the verbs changes, they all change, so having "Move" and "Adjust" in the same dialog means the common code would not be used, adding time and cost. When gathering examples from other sims, I noticed some used, "Move," while others used, "Adjust," between sims, but I didn't notice these verbs are all the same within a single sim's dialog.

Today, we felt a good solution would be to customize each screen's keyboard dialog, but use the common code to not only make the implementation more efficient, but also be able to name the things users can move per screen. The only downside to this solution is the notion of combining objects that "Move" versus "Adjust" into a single verb. The upside is efficiency in implementation, consistency with other sims' dialogs, and the common code caught things I had forgotten in my mockups, such as End for 'Move in larger steps'.

This takes us to the second issue to think about, which is the implementation. I don't feel knowledgeable or that it's my realm to discuss the use of the AccessibleSlider, so I this is likely a conversation still to be had. In the meantime, I'm comfortable with the design solution and have a much better understanding of how the common code for this dialog works.

Here's an example of what the Median screen's dialog will look like using common code and implementing our solution:

For the Mean & Median screen, the first line on the left will say, "Move soccer ball, predict tools," and for the Variability screen, "Move soccer ball, interval tool"

I have a question about people using this dialog box. Would anyone be confused if the first line said, "Move soccer ball, adjust interval tool" ? I'm ok with the word, "adjust" isn't there, but thought I'd ask as it might be a good solution to have both verbs in there while still using common code.

[samreid]

I opened a side issue https://github.com/phetsims/center-and-variability/issues/367 about whether the implementation should extend AccessibleSlider or AccessibleValueHandler and @jessegreenberg pointed out the other alternative KeyboardDragListener. For the keyboard help dialog, it seems we are ready to implement the design above.

[samreid]

On hold until we finish #351

[catherinecarter]

Adding that we will need to add the jump feature to the keyboard dialog. This will be similar to Ratio and Proportion where the user presses 0-9 to jump a ball to its desired location when a ball has been selected.

0 = 10 1-9 = their location on the number line.

Documentation in the keyboard will say something like this: Jump ball to number line location (0 jumps to 10)

Ratio and Proportion example:

[marlitas]

I believe this issue can now be taken off hold since most of the new alt-input work is complete. Sending over to @catherinecarter to provide the final mock-up for this.

[samreid]

Keep in mind we support home/end, and there is an open issue #437 about whether we should support page up/page down.

[catherinecarter]

Here are a couple of screenshots so asynchronous feedback can be made. Thank you!

Idea for global dialog box

Idea for per-screen dialog boxes

• Median screen
  

• Median & Mean screen

• Variability screen

[terracoda]

I'll have a look at this asap :-)

[terracoda]

There's a lot here, I'll take it one interaction at a time. First the ball and card interaction. It is good to break things up into steps, so adding an example from BASE

I think for the balls and cards it might work well if you just mention how they are grabbed, and then how they are moved when they are grabbed:

Grab or Release Ball or Card

Move Grabbed Ball or Card

Move grabbed ball or card [<][>] or [A][D]

Question: Do you have any other shortcuts or actions implemented for moving slower or jumping around? If not, remove all the information about sliders and the additional shortcuts, including jumping to Home and End.

In addition maybe If you think you need to describe how to get to the groups and then to a ball or card, you can add the following before the grabbing and moving sections. But This is technically included in Basic Actions and is well supported by the visual hint.

Choose a Ball or Card

Move to group [Tab] Move to ball or card [<][>]

[terracoda]

Oh, I see the tools that operate like sliders. I will review more closely :-)

[terracoda]

My thoughts

Median Screen

• You can only Predict the Median.
• To change or adjust the Median you have to grab and move a soccer ball. The red down arrow only shows the median. You you do not need "Adjust median" in the instructions, only "Predict Median".

I suggest something like this if it make sense to you:

Predict Median

• Move prediction [<][>] (Would this be 1 full number position?)
• Move in smaller steps [Shift] + [<][>] (The mean can have really small steps, so you may need to define big and small steps based on the needs for the mean pointer?)
• Move in larger steps [Pg Up][<Pg Dn] (usually this is total positions divided by 10)
• Jump to start of number line [Home]
• Jump to end of number line [End]

Median & Median Screen

I suggest something like this if it make sense to you:

Predict Median or Median

(Same instructions keeping in mind if there are any special differences for small or large steps for median and mean?)

Variability Screen

Question: What is the point of the pointer? Would something like this make sense?

Mark Important Areas

• Move pointer or interval handles [<][>]
• Move in smaller steps [Shift] + [<][>]
• Move in larger steps [Pg Up][<Pg Dn]
• Jump to start of number line [Home]
• Jump to end of number line [End]

[terracoda]

For pointer and interval handles, I like "move" instead of "adjust" because it is "move" is simpler to me, but the verb "adjust" is fine if you think it is a better word. I think "Adjust position" might be clearer than just "adjust" since you are not adjusting a mathematical value - just moving a visual marker.

[catherinecarter]

Here are new mockups with language specific to each screen. @terracoda, please let us know if this hits the mark :) Thank you!

Median screen

Mean & Median screen

Variability screen

[emily-phet]

Suggestion: In second screen, replace "move predictions" with "move prediction pointers". "Move predictions" may read as odd. I love "pointer" being used in Screen 3, so that addition of that in screen 2 language might help.

Suggestion: Remove "shaded" from heading in screen 3. I think "Interval Block" is descriptive enough for me.

Also note, @catherinecarter, you'll want to make sure this is language that is propagated throughout the sim's materials. So, in the code, in supporting documentation, in teacher tips, in primer videos, etc. If there's anything that needs to have a different name in a different context, now is the time to get all those in sync! From experience, it's a bummer when you notice that the sound design doc, the sound primer for teachers, and other materials use different names for the same thing...

When you work on your next sim, you can start the naming process early so everything starts and stays consistent. But for now, it's something to check in on as language about the sim is being made public.

[terracoda]

I agree with Emily about removing the "s" on "Move predictions". There's only one at a time. And removing "shaded" too. I wasn't 100% sure that was needed when I suggested it on slack.

I also think it would be be good to just try the general language of "steps", "smaller steps", and "bigger steps" and let learners figure out that for some of the tools the steps are different.

It will make the shortcuts easier to read (and hear). I am not sure how important it is that the learner knows in advance the exact size of the step. They will see (and eventually hear) the step size when they interact.

Do you think the general simplified step sizing would work?

[samreid]

@catherinecarter is this ready for development?

[terracoda]

One more comment, since the interactions are all horizontal, you may only want to present the Left and Right Arrows keys for all the pointer tools. The Up and Down arrows will always work because the interactions are like sliders under the hood. But it is not necessary to cue them, so if you want to make dialog less busy, you could remove the extra Up Down cues.

[catherinecarter]

Yes. I've implemented @emily-phet and @terracoda's suggestions:

Median screen

Mean & Median screen

Variability screen

[samreid]

I have no experience in developing these, so it seems like a good learning opportunity.

[samreid]

Plain text of the above so far to help with comparison and development (still need 3rd section.) Not the updated wording regarding "largest" etc.

Grab or Release Ball or Card
- Grab or release: Space or Enter

Move Grabbed Ball or Card
- Move grabbed ball or card: <- -> or AD
- Move in larger steps: PgUp PgDn
- Jump to start of cards or number line: Home
- Jump to end of cards or number line: End
- Jump ball to tick mark: 0-9

Predict Median
- Move predict median: <- -> or UP DOWN
- Move in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End

__________________

Grab or Release Ball or Card
- Grab or release: Space or Enter

Move Grabbed Ball
- Move grabbed ball: <- -> or AD
- Move in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End
- Jump ball to tick mark: 0-9

Predict Mean or Median 
- Move prediction pointer: <- -> or UP DOWN
- Move prediction pointer in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End

[samreid]

Do we need a help section about "arrow keys to select within a group"?

[catherinecarter]

The "arrow keys to select within a group" is in the basic actions, and there will be a cue on the screen to also indicate how to perform the action.

For the Variability screen, a couple changes.

1. In the "Move Pointer, Interval Handles, or Interval Block" section, let's just say "Move" instead of "Move object." That seems to be how some of the other sims are handling the direction to move the objects listed in the title
2. Move the "Move in smaller steps" to be above the "Move in larger steps" Note: let's leave the order the as is for the Mean/Median screen since the top two ("Move prediction pointer" and "move prediction pointer in larger steps" are both referring to the pointers (both of the them) while the "smaller steps" direction only refers to the predict mean since the predict median does not move in smaller steps)
3. As @samreid noted, some of the mockups say "bigger" and some say "larger." The consistent word from other sims is "larger." Sorry about that!

Everything else seems good to me. Thanks!

[samreid]

For my reference:

Sam Reid What is the best place to look for an exemplar for creating our first Keyboard Help Dialog?

Jesse Greenberg Hi! Maybe a recent one like QuadrilateralKeyboardHelpContent. The basic building blocks are KeyboardHelpSection and KeyboardHelpSectionRow. Let me know if you have questions!

Sam Reid Thanks, how does the keyboard help dialog “demo” compare?

Jesse Greenberg It is also good, looks like it demos premade common code sections, icons, and layout.

[samreid]

I tried to incorporate the comments above into this text:

Grab or Release Ball or Card
- Grab or release: Space or Enter

Move Grabbed Ball or Card
- Move grabbed ball or card: <- -> or AD
- Move in larger steps: PgUp PgDn
- Jump to start of cards or number line: Home
- Jump to end of cards or number line: End
- Jump ball to tick mark: 0-9

Predict Median
- Move predict median: <- -> or UP DOWN
- Move in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End

__________________

Grab or Release Ball
- Grab or release: Space or Enter

Move Grabbed Ball
- Move grabbed ball: <- -> or AD
- Move in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End
- Jump ball to tick mark: 0-9

Predict Mean or Median
- Move prediction pointer: <- ->
- Move prediction pointer in larger steps: PgUp PgDn
- Move prediction pointer in smaller steps: Shift + <- ->
- Jump to start of number line: Home
- Jump to end of number line: End

__________________

Grab or Release Ball
- Grab or release: Space or Enter

Move Grabbed Ball
- Move grabbed ball: <- -> or AD
- Move in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End
- Jump ball to tick mark: 0-9

Move Pointer, Interval Handles, or Interval Block
- Move: <- ->
- Move in smaller steps: Shift + <- ->
- Move in larger steps: PgUp PgDn
- Jump to start of number line: Home
- Jump to end of number line: End

[samreid]

I think I'm at a good point to check in for design and code review.

For code review:

• [x] Are the string keys and nesting as we want it?
• [x] Is the reuse of strings at the appropriate level?
• [x] Is it supposed to be nested under a11y so it won't show in rosetta yet?

For design review:

• [x] Please check that it mostly matches expectations. I don't have explicit control over certain things like word where a line break occurs, so please keep that in mind as giving feedback. Mostly check that the content seems correct.

[samreid]

I added a few more commits to make it even more ready for review.

[catherinecarter]

Looks really great! Each screen gives customized instructions for the keyboard and describes how to interact. Good on my end.

[matthew-blackman]

The help dialog classes look excellent and the hierarchy makes sense to me. Nice work @samreid!

I reached out to the a11y team about how to organize strings and received the following info from @jessegreenberg:

Everything under the a11y key is not available for translation. Strings that are only used for Interactive Description and Voicing are hidden from translators for now. But all strings that appear visually in Text most likely need to be translatable. I think that means you should move everything out from the a11y key for CaV.

The above commits remove the a11y level of the string organization so that the keyboard help dialog will be translateable via Rosetta. Things are looking good here. Closing.