Useful Keyboard Shortcuts to check shape and reset shape

[terracoda]

I am wondering if we really need the Corner Labels checkbox. It was decided earlier on to have it on by default which I think is good.

Given, that the labels/names cannot be hidden for voicing or description, and we have 5 other controls now to support different aspects of shape exploration, I would like to review the importance/the need for visually hiding the Corner Labels for the visual mode.

I do see that it might be easier to align the corners on the grid without the letter there. Is that the only reason?

[brettfiedler]

Given the sandbox nature of the sim, I just assumed we offer up the option to hide the labels for any use case a teacher may have to do so. Perhaps, depending on letter translation to non-Latin (#218) and/or if the alphabet isn't culturally used for geometry, some classrooms may prefer it off? I don't feel strongly about it.

There aren't many controls though compared to most of the math sims, so unless you feel it clutters the interface?

[terracoda]

My main concern is how to do groups of controls in the Voicing design. For keyboard users, having shortcuts to jump to a group would be very handy, but would need some global thinking for all sims.

We currently have 16 Tab stops in the sim not including the Voicing Toolbar or the bottom Navbar.

[brettfiedler]

Is the "shortcut to jump to a group" something you'd like to try out for Quad? Especially since those 4 checkboxes are pretty useless for the non-visual experience, perhaps we could group them together? Maybe move the Corner Label to the bottom of the list in that group as well?

[terracoda]

Yes. Another thing we could try is shortcuts for useful things - like Reset Shape, turn music on or off.

[terracoda]

It might be helpful to list out what sims of custom keyboard shortcuts:

1. BASE - to jump the balloons around. The WASD keys are common code for custom movable objects and already work for moving Quad's corners and sides around.
2. MAL - To pause or play action in Observation Window Alt + K, and Alt + L to Step forward in bigger steps when sim is paused. I think these are common code shortcuts for Timing Control Node.
3. RaP - to align the hands on a grid line
4. FL - Auto-slide magnet option

Listing out sims may not be a useful exercise :-)

[terracoda]

To follow-up on https://github.com/BLFiedler)... grouping

We already have grouping patterns we use in the PDOM (e.g. using some combo of heading aria-role=group, and possibly aria-labelledby?) I think we would want to structure things in the same way as we already do, but of course, we wouldn't get the code semantics spoken aloud for free in Voicing.

We have keyboard shortcut instructions for navigating groups with the arrow keys, so that's covered.

Group benefits for Quad? Grouping the visual guides into a single group of controls would significantly reduce the number of Tab Stops in this sim making it more efficient to get from a Corner or Side to say, the Shape Sounds checkbox. This would likely be beneficial to all keyboard users.

Possible group headings: "Shape Guides", "Visual Guides", or just "Guides" We could consider Keyboard Shortcuts for visual group headings or visually organized groups, or we could just keep relying on our regular pattern of using the Tab Key and the Arrow keys with the group.

For Voicing the visual group heading would likely need to be a reading block.

Example Layout with group focus highlights

Visual Comparison from BASE's Radiogroup

[terracoda]

We could also consider making a very general hint about the visual guides, and adding it to the Group heading reading block rather than having a hint on each of the check boxes.

So if Helpful Hints is checked, we would hear: Guides, Create shapes with or without help of visual guides.

Reading block focus or mouse over - not actively reading

[jessegreenberg]

I don't believe grouping is the right thing for checkboxes. Radio buttons are grouped because that is the convention and the browser does that for us for free. Radio button groups let you change one value from several options so it makes sense to me that the group is one stop in the traversal order. Checkboxes each control a unique value so each one gets a stop in the traversal order. If we need to group them we will have to implement that ourselves and work around the default browser behavior.

[terracoda]

@jessegreenberg, I agree that we have not yet created a group of checkboxes for a PhET Sim, but checkbox interactions of a similar type or theme can be grouped together.

There isn't an equivalent native HTML group element like radiogroup for checkboxes, but there are headings and list structures, and legend and fieldset elements that can be used to group things of a similar theme or type together. https://www.w3.org/WAI/tutorials/forms/grouping/#checkboxes

These types of structures natively create efficient keyboard navigation for screen reader users.

@BLFiedler and I were brainstorming ideas on how to make keyboard navigation more efficient in this sim for the Voicing Feature which does not have native access to PDOM headings in the same way as screen reader does. I have a feeling this could be helpful down the road for many sims - sims that have a number of grouped interactions.

I feel like the "visual guides" are semantically part of a group, hence the ideas above.

Maybe a group navigation shortcut would be more general. If we can do that a learner could use the shortcut to jump to the first member of a group if a group exists?

[brettfiedler]

Should we consider reordering the checkboxes for the most relevant to non-visual to the top and just jump to the top checkbox?

• Angles
• Grid
• Diagonals
• Labels (default on)

[brettfiedler]

@jessegreenberg , go ahead and reorder these anywho

• [x] Reorder checkboxes according to previous comment.

Angles may change name, but this can happen first.

[brettfiedler]

@terracoda will investigate a shortcut combination to the first checkbox and a description for the dialog.

[jessegreenberg]

Checkboxes have been re-ordered, removing my assignment.

[terracoda]

I reviewed the shortcuts we have in sims with Alt Input:

• [Alt] + [K], pause or play action (MAL)
• [Alt] + [L], when paused step forward in bigger steps (MAL)
• [1][2][3] Auto-slide Bar Magnet (FL)
• [J] + [C] Jump start of ruler to center of m1 sphere (GFL)
• [J] + [H] Jump and release ruler home position (GFL)
• [W][A][S][D] - move custom objects up, left, down, right (many examples)
• [W][S] / [Up][Down] - bi-manual input for custom Both Hands interaction (RaP)
• [0] - [9] - jump both hands to same tick mark (RaP)
• [J] + [S] - jump close to sweater (BASE)
• [J] + [W] - jump to wall (BASE)
• [J] + [N] - jump to near wall (BASE)
• [J] + [C] - jump to center (BASE)
• [Alt] + [C] - check your answer (game screen of FMW)
• [J] - jump a ruler or marker control through a series of locations (GO)

Useful for Quad and General Use?

• [Alt] + [C] - check shape name (inspiration from FMW)
• [Alt] + [R] - reset shape or do a partial reset, many sims have a partial reset.

If we have a check shape and reset shape short cut, we may not need to worry about the number of check boxes. That said, I propose either of these for Quad:

• [J] + [X] - jump to next checkbox, or jump to first checkbox

• [Shift] + [J] + [X] jump to previous checkbox. I am not proposing we have a jump to last checkbox.

• [J] + [G] - jump to first thing in a special group (Tab key instead of arrows would move between between special group items)

• [Shift] + [J] + [G] - jump to first thing in the previous special group

Any thoughts @BLFiedler ?

[terracoda]

@BLFiedler, we were originally wondering if we had too many check boxes and how to navigate efficiently through them and the other controls, but maybe Keyboard Shortcuts for generally really useful things would be a better approach?

Things I think learners will want to do are:

• check their shape or their shape's key properties (depending on the visibility of the shape name), and
• reset the shape

Jumping to a checkbox would certainly be useful for this sim, but it is less generalizable across all sims.

[terracoda]

@jessegreenberg, your thoughts are welcome, too.

[pixelzoom]

@terracoda said in Slack#general:

For people who would like to contribute to the idea of a general keyboard shortcut for a "partial reset", please comment in https://github.com/phetsims/quadrilateral/issues/213#issuecomment-1293628990

My 2 cents: A parital reset should be the same shortcut as a full reset, but with an additional modifier key. This is often the purpose of modifier keys -- to change the behavior (constrain, expand, etc.) another similar shortcut.

Also keep in mind that a sim may have more than 1 thing that can be partially reset. In those cases, if there's a standard shortcut for "partial reset", what does it apply to? And what is the shortcut for the other things that need a "partial reset"?

[kathy-phet]

I agree with @pixelzoom 's suggestion here, and was wondering @terracoda if there is a hot key for full reset? I don't see one on your list. Seems like a nice add for a 'reset all' shortcut for the "Basic Action" list in the keyboard help dialog?

Maybe Ctrl- R and Ctrl-Shift-R? or something like that?

[terracoda]

I did notice that we did not have a general reset all short cut. Honestly, the need for shortcut keys for screen reader users has been relatively rare - mostly needed for quite specific interactions and sim scenarios - likely because the sims already work well with built-in screen reader shortcuts. For example, screen reader users do not need to Tab through all interactive elements to get to the rest all button. They can easily look for buttons specifically, efficiently skipping over other interactions. They can do that with all interactions.

However, for Voicing, learners do not have that flexibility. It’s worth thinking about the need for shortcuts more generally.

I also like the idea of a modifier key.

[terracoda]

Ctrl + R is already the shortcut that re-loads the page in most if not all internet browsers. This of course, would also reset the sim. I like to avoid the Control key as a single modifier as mode browser shortcuts start with that modifier key.

I suggest we use R since what we want to do is something similar (refresh or re-load some aspect of the sim). In addition, we could stick with Alt and Shift keys as modifiers since we already use those modifier keys in sims.

So something like this:

• Alt + Shift + R - a partial reset option. I say option because it seems different sims can design different kinds of partial resets.
• Alt + R- as a general Reset All shortcut. This allows visual keyboard learners to quickly reset the sim without having to tab through to the reset all button.

Alternatively, we could use Ctrl and Alt together, but not Ctrl on its own like this:

• Ctrl + Alt + R a partial reset option.
• ALT + R as a general Reset All shortcut.

My main design question is which reset would we like to make more difficult to accomplish? Pressing 3 keys is more difficult than pressing 2. I think it makes sense to make the partial reset the simpler pattern.

• Alt + R - a partial reset option.
• Alt + Shift + R- as Reset All shortcut. Having to press 3 keys here means they really want to do a full reset.

[terracoda]

Either way is fine for me, really.

[terracoda]

@jessegreenberg, should I put general shortcuts in a different repo?

What I am proposing for Quad in this issue is:

• [Alt] + [C] - for check shape name
• [Alt] + [R] or [Alt + Shift + R ] - for reset shape depending on what the team prefers for a partial reset.

I think we should think about my proposed jump keys like [J] + [X] and [J] + [G] more generally in case we want a set of shortcuts for the Voicing feature, that would create a more robust navigation experience.

[terracoda]

My only remaining question for this issue is whether the partial reset or the full reset should require 3 keys?

[terracoda]

Maybe we can close this issue, @pixelzoom pointed out that a partial reset should be the shortcut that adds to the general shortcut for a full reset.

[brettfiedler]

I believe this has been settled in requests for #249. Re-open if that is not the case.

[terracoda]

I think this issue was closed prematurely.

Re-opening so sim-specific keyboard shortcuts can be implemented.

I think implementation of the shortcuts might need to be separate from the creation of the dialog that documents them.

The sim-specific shortcuts for this sim are:

What I am proposing for Quad in this issue is:

• [Alt] + [C] - for check shape name
• [Alt + Shift + R ] - for reset shape

The general reset sim shortcut, [Alt] + [R] , is in a scenery issue.

[jessegreenberg]

[Alt] + [C] - for check shape name

@terracoda how does this shortcut behave?

[terracoda]

From https://github.com/phetsims/quadrilateral/issues/213#issuecomment-1293646475

Things I think learners will want to do are:

- check their shape or their shape's key properties (depending on the visibility of the shape name), and
- reset the shape

Jumping to a checkbox would certainly be useful for this sim, but it is less generalizable across all sims.

I think the above means that I would like to hear the following responses when a learner "checks their shape". This is the response that fires when the a learner toggles from "Shape Name Hidden" to showing the current shape's name.

• When shape name is shown, and a learner presses [Alt] + [C] we hear the response:
  • "You have a {{ShapeName}}."
• When shape name is hidden, and a learner presses [Alt] + [C] we hear the the shape's key properties response. This is the response that fires when shape name is hidden and a learner creates a new shape. This is not the shape-maintenance response. Here's a few examples:
  • "Opposites in parallel." (parallelogram)
  • "All right angles. All sides equal." (square)
  • "All right angles." (rectangle)
  • "All sides equal." (rhombus)
  • "Corners {{CornerLetter}} and {{CornerLetter}} are equal and pairs of adjacent sides are equal. (kite)
  • "No sides in parallel." (convex quadrilateral)

This shortcut allows a learner to keep their attention on the shape, and provides is easy access to repeating information that is delivered in passing during interaction, and then is also visually available for learners who are able to process the visual information. That is learners who can easily read the displayed shape name, or learners who can visually assess the properties of the current shape by looking at it.

I think this shortcut needs to be Voicing specific.

Over to you @jessegreenberg,apologies for the delay.

[jessegreenberg]

https://github.com/phetsims/quadrilateral/issues/213#issuecomment-1362862276

I don't have easy access to that description unfortunately, Ill have to come back and do some refactoring.

reset the shape

@terracoda what should the Voicing response be when these keys are pressed? Should it be the full response of the "Reset Shape" button (including the name)? Or just its context response?

[terracoda]

@jessegreenberg, can the Reset Shape shortcut voice the same responses as the Reset Shape button?

• Voice the Name Response if Context Responses is not checked in Preferences, and
• Voice both Name & Context Response if Context Responses is checked in Preferences.

[jessegreenberg]

I am running into problems with KeyboardListener supporting both a hotkey for Reset All and Reset Shape. Those two listeners have an overlap in keys and both fire. They are two different listeners at very different places in the code and there isn't a good way to choose which one should fire yet. We need a better way to support this

There is a lot still on the plate for publishing quadrilateral. I am wondering if we should put this on hold and add this new feature if we have extra time.

[jessegreenberg]

Maybe the right behavior is that a set of keys should fire the listener if ONLY those keys are down. That could simplify things here and in KeyboardListener. I have previously been assuming that keys should fire whenever the required keys are down, but others could be pressed too.

[jessegreenberg]

OK, this is done mostly by the above commit and https://github.com/phetsims/scenery-phet/issues/789.

@terracoda can you please review this issue and that scenery-phet issue and confirm everything is as you expect?

[terracoda]

Testing in progress

Oops, the visual hints are sticky after a full reset using the hot keys Option + R

[terracoda]

[jessegreenberg]

Thanks for finding! That is a keyboard specific problem because the isPressedProperty for the interaction hint doesn't get set until focus moves.

[terracoda]

Yep, it only happens if I do not move focus after using a Option + R to reset.

[jessegreenberg]

I think that is fixed by the above commit.

[terracoda]

Ok, will pull again :-)

[terracoda]

Hmm, not yet fixed. The sim crashed.

[terracoda]

Much better. I think you fixed it.

All the shortcuts seem to working well. Thanks @jessegreenberg.