phetsims / density-buoyancy-common

Common code for the Density and Buoyancy simulations
GNU General Public License v3.0
0 stars 2 forks source link

Blocks should use GrabDragInteraction for keyboard input #209

Closed zepumph closed 1 month ago

zepumph commented 3 months ago

From https://github.com/phetsims/density-buoyancy-common/issues/190, because tabbing to the blocks makes the gravity no longer effect the block, a "tab then grab" interaction seems best. We also, in design meeting, discussed how this case would be similar to ESP where a skater is moving in a half pipe. Tabbing to the skater shouldn't pause the skater. This is a overload of the focus() event, and so GrabDragInteraction is likely a good solution here. Let's take a look.

zepumph commented 2 months ago

This is getting us in the right direction, but still has some work to do and review, like the TODOs in it.

```diff Subject: [PATCH] opt out of overwriting range in MaterialControlNode when there is a density slider that already has a range, https://github.com/phetsims/density-buoyancy-common/issues/268 --- Index: js/common/view/MassView.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/common/view/MassView.ts b/js/common/view/MassView.ts --- a/js/common/view/MassView.ts (revision 24f7b3b7029f560b811fff0e156742ee41b20980) +++ b/js/common/view/MassView.ts (date 1722029607182) @@ -23,6 +23,7 @@ import MassThreeMesh from './MassThreeMesh.js'; import { THREEModelViewTransform } from './DensityBuoyancyScreenView.js'; import sharedSoundPlayers from '../../../../tambo/js/sharedSoundPlayers.js'; +import GrabDragInteraction from '../../../../scenery-phet/js/accessibility/GrabDragInteraction.js'; const INVERT_Y_TRANSFORM = ModelViewTransform2.createSinglePointScaleInvertedYMapping( Vector2.ZERO, Vector2.ZERO, 1 ); @@ -110,30 +111,14 @@ focusable: true } ); const endKeyboardInteraction = () => { - keyboardDragListener.interrupt(); - mass.interruptedEmitter.removeListener( endKeyboardInteraction ); - this.focusablePath!.removeInputListener( blurListener ); + grabDragInteraction.interrupt(); + + // TODO: is it best to be graceful here? OR try to figure out why we are removing the listener twice? https://github.com/phetsims/density-buoyancy-common/issues/209 + mass.interruptedEmitter.hasListener( endKeyboardInteraction ) && mass.interruptedEmitter.removeListener( endKeyboardInteraction ); releaseSoundPlayer.play(); mass.endDrag(); }; - const blurListener = { - blur: endKeyboardInteraction - }; - - this.focusablePath.addInputListener( { - focus: () => { - - // We want the newer interaction to take precedent, so tabbing to the item should interrupt the previous mouse drag (if applicable). - mass.userControlledProperty.value && mass.interruptedEmitter.emit(); - - mass.interruptedEmitter.addListener( endKeyboardInteraction ); - this.focusablePath!.addInputListener( blurListener ); - grabSoundPlayer.play(); - mass.startDrag( mass.matrix.translation ); - } - } ); - const keyboardDragListener = new KeyboardDragListener( { // In model units per second dragSpeed: 3, @@ -148,9 +133,27 @@ tandem: Tandem.OPT_OUT } ); - this.focusablePath.addInputListener( keyboardDragListener ); + const grabDragInteraction = new GrabDragInteraction( this.focusablePath, keyboardDragListener, { + grabCueOptions: { // TODO: Position the cue? https://github.com/phetsims/density-buoyancy-common/issues/209 + centerX: 0, + bottom: 0 + }, + onGrab() { + // We want the newer interaction to take precedent, so tabbing to the item should interrupt the previous mouse drag (if applicable). + mass.userControlledProperty.value && mass.interruptedEmitter.emit(); + + mass.interruptedEmitter.addListener( endKeyboardInteraction ); + grabSoundPlayer.play(); + mass.startDrag( mass.matrix.translation ); + }, + onRelease() { + endKeyboardInteraction(); + }, + tandem: Tandem.OPT_OUT + } ); this.disposeEmitter.addListener( () => { + grabDragInteraction.dispose(); keyboardDragListener.dispose(); this.focusablePath && this.focusablePath.dispose(); } );
samreid commented 2 months ago

@AgustinVallejo and I also wrote to @jessegreenberg on slack:

Hi @jessegreenberg. I’m working with @AgustinVallejo on https://github.com/phetsims/density-buoyancy-common/issues/209 where we would like to use GrabDragInteraction to publish Buoyancy 1.0. We opened https://github.com/phetsims/scenery-phet/issues/863 to convert it to TypeScript. While we were there, we also saw several issues related to GrabDragInteraction: https://github.com/phetsims/scenery-phet/issues?q=is%3Aissue+is%3Aopen+grabDragInteraction. Some are related to voicing which is not in the purview for Buoyancy 1.0. Others we wanted to ask you about--do we need to do anything to get ready for production?

samreid commented 1 month ago

@jessegreenberg identified https://github.com/phetsims/scenery-phet/issues/708 as potentially related to GrabDragInteraction for our upcoming publication, so we added it to the project board.

samreid commented 1 month ago

The initial focus area is offset:

image
samreid commented 1 month ago

Minor refactoring and experimentation:

```diff Subject: [PATCH] opt out of overwriting range in MaterialControlNode when there is a density slider that already has a range, https://github.com/phetsims/density-buoyancy-common/issues/268 --- Index: js/common/view/MassView.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/common/view/MassView.ts b/js/common/view/MassView.ts --- a/js/common/view/MassView.ts (revision f16281bba8465ed9b926bc5292581830e3cee76c) +++ b/js/common/view/MassView.ts (date 1723039202920) @@ -23,6 +23,7 @@ import MassThreeMesh from './MassThreeMesh.js'; import { THREEModelViewTransform } from './DensityBuoyancyScreenView.js'; import sharedSoundPlayers from '../../../../tambo/js/sharedSoundPlayers.js'; +import GrabDragInteraction from '../../../../scenery-phet/js/accessibility/GrabDragInteraction.js'; const INVERT_Y_TRANSFORM = ModelViewTransform2.createSinglePointScaleInvertedYMapping( Vector2.ZERO, Vector2.ZERO, 1 ); @@ -30,13 +31,12 @@ public readonly massMesh: MassThreeMesh; - private readonly positionListener: () => void; - private readonly massTagNode: Node | null = null; protected readonly tagOffsetProperty: Property = new Property( Vector3.ZERO ); - public readonly focusablePath: Path | null; + public readonly focusablePath: Path | null = null; public readonly focusableShapeProperty = new Property( new Shape() ); + private readonly grabDragInteraction: GrabDragInteraction | null = null; protected constructor( public readonly mass: Mass, initialGeometry: THREE.BufferGeometry, @@ -61,7 +61,7 @@ const grabSoundPlayer = sharedSoundPlayers.get( 'grab' ); const releaseSoundPlayer = sharedSoundPlayers.get( 'release' ); - this.positionListener = () => { + const positionListener = () => { const position = mass.matrix.translation; // LHS is NOT a Vector2, don't try to simplify this @@ -98,8 +98,6 @@ this.massTagNode && repositionMassTagNode(); }; - this.focusablePath = null; - if ( mass.canMove ) { this.focusablePath = new InteractiveHighlightingPath( this.focusableShapeProperty, { @@ -108,30 +106,14 @@ focusable: true } ); const endKeyboardInteraction = () => { - keyboardDragListener.interrupt(); - mass.interruptedEmitter.removeListener( endKeyboardInteraction ); - this.focusablePath!.removeInputListener( blurListener ); + this.grabDragInteraction!.interrupt(); + + // TODO: is it best to be graceful here? OR try to figure out why we are removing the listener twice? https://github.com/phetsims/density-buoyancy-common/issues/209 + mass.interruptedEmitter.hasListener( endKeyboardInteraction ) && mass.interruptedEmitter.removeListener( endKeyboardInteraction ); releaseSoundPlayer.play(); mass.endDrag(); }; - const blurListener = { - blur: endKeyboardInteraction - }; - - this.focusablePath.addInputListener( { - focus: () => { - - // We want the newer interaction to take precedent, so tabbing to the item should interrupt the previous mouse drag (if applicable). - mass.userControlledProperty.value && mass.interruptedEmitter.emit(); - - mass.interruptedEmitter.addListener( endKeyboardInteraction ); - this.focusablePath!.addInputListener( blurListener ); - grabSoundPlayer.play(); - mass.startDrag( mass.matrix.translation ); - } - } ); - const keyboardDragListener = new KeyboardDragListener( { // In model units per second dragSpeed: 3, @@ -146,16 +128,38 @@ tandem: Tandem.OPT_OUT } ); - this.focusablePath.addInputListener( keyboardDragListener ); + // const pt = modelViewTransform.modelToViewPoint( mass.matrix.translation.toVector3().plus( this.tagOffsetProperty.value ).plusXYZ( 0, 0, 0.0001 ) ); + + this.grabDragInteraction = new GrabDragInteraction( this.focusablePath, keyboardDragListener, { + // grabCueOptions: { // TODO: Position the cue? https://github.com/phetsims/density-buoyancy-common/issues/209 + // center: pt + // }, + onGrab() { + // We want the newer interaction to take precedent, so tabbing to the item should interrupt the previous mouse drag (if applicable). + mass.userControlledProperty.value && mass.interruptedEmitter.emit(); + + mass.interruptedEmitter.addListener( endKeyboardInteraction ); + grabSoundPlayer.play(); + mass.startDrag( mass.matrix.translation ); + }, + onRelease() { + endKeyboardInteraction(); + }, + tandem: Tandem.OPT_OUT + } ); this.disposeEmitter.addListener( () => { + this.grabDragInteraction!.dispose(); keyboardDragListener.dispose(); this.focusablePath && this.focusablePath.dispose(); } ); } - this.mass.transformedEmitter.addListener( this.positionListener ); - this.positionListener(); + this.mass.transformedEmitter.addListener( positionListener ); + + this.disposeEmitter.addListener( () => this.mass.transformedEmitter.removeListener( positionListener ) ); + + positionListener(); } // Override in subclasses to add subclass-specific behavior @@ -172,10 +176,11 @@ public decorate( decorationLayer: MassDecorationLayer ): void { this.massTagNode && decorationLayer.massTagsLayer.addChild( this.massTagNode ); + // this.focusablePath && this.focusablePath.detach(); + // this.focusablePath && decorationLayer.forceDiagramLayer.addChild( this.focusablePath! ); } public override dispose(): void { - this.mass.transformedEmitter.removeListener( this.positionListener ); this.massMesh.dispose(); this.massTagNode && this.massTagNode.dispose(); ```

I cannot get the text cue to display. @jessegreenberg are you available to assist?

samreid commented 1 month ago

@marlitas helped us get even further, now the cue is showing and the dotted line is showing:

```diff Subject: [PATCH] opt out of overwriting range in MaterialControlNode when there is a density slider that already has a range, https://github.com/phetsims/density-buoyancy-common/issues/268 --- Index: density-buoyancy-common/js/common/view/MassView.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/density-buoyancy-common/js/common/view/MassView.ts b/density-buoyancy-common/js/common/view/MassView.ts --- a/density-buoyancy-common/js/common/view/MassView.ts (revision 655a83232baa629581daa76890ac6b06e23f121f) +++ b/density-buoyancy-common/js/common/view/MassView.ts (date 1723067653928) @@ -23,6 +23,7 @@ import MassThreeMesh from './MassThreeMesh.js'; import { THREEModelViewTransform } from './DensityBuoyancyScreenView.js'; import sharedSoundPlayers from '../../../../tambo/js/sharedSoundPlayers.js'; +import GrabDragInteraction from '../../../../scenery-phet/js/accessibility/GrabDragInteraction.js'; const INVERT_Y_TRANSFORM = ModelViewTransform2.createSinglePointScaleInvertedYMapping( Vector2.ZERO, Vector2.ZERO, 1 ); @@ -30,13 +31,12 @@ public readonly massMesh: MassThreeMesh; - private readonly positionListener: () => void; - private readonly massTagNode: Node | null = null; protected readonly tagOffsetProperty: Property = new Property( Vector3.ZERO ); - public readonly focusablePath: Path | null; + public readonly focusablePath: Path | null = null; public readonly focusableShapeProperty = new Property( new Shape() ); + private readonly grabDragInteraction: GrabDragInteraction | null = null; protected constructor( public readonly mass: Mass, initialGeometry: THREE.BufferGeometry, @@ -61,7 +61,7 @@ const grabSoundPlayer = sharedSoundPlayers.get( 'grab' ); const releaseSoundPlayer = sharedSoundPlayers.get( 'release' ); - this.positionListener = () => { + const positionListener = () => { const position = mass.matrix.translation; // LHS is NOT a Vector2, don't try to simplify this @@ -92,46 +92,35 @@ this.focusableShapeProperty.value = shape; - this.focusablePath.focusHighlight = shape; + // debugger; + // this.focusablePath.focusHighlight.shape = shape; + if ( this.focusablePath.focusHighlight ) { + + // TODO: type error + this.focusablePath.focusHighlight.setShape( shape ); + } } this.massTagNode && repositionMassTagNode(); }; - this.focusablePath = null; - if ( mass.canMove ) { + this.focusablePath = new InteractiveHighlightingPath( this.focusableShapeProperty, { accessibleName: this.mass.accessibleName, tagName: 'div', focusable: true } ); const endKeyboardInteraction = () => { - keyboardDragListener.interrupt(); - mass.interruptedEmitter.removeListener( endKeyboardInteraction ); - this.focusablePath!.removeInputListener( blurListener ); + this.grabDragInteraction!.interrupt(); + + // TODO: is it best to be graceful here? OR try to figure out why we are removing the listener twice? https://github.com/phetsims/density-buoyancy-common/issues/209 + mass.interruptedEmitter.hasListener( endKeyboardInteraction ) && mass.interruptedEmitter.removeListener( endKeyboardInteraction ); releaseSoundPlayer.play(); mass.endDrag(); }; - const blurListener = { - blur: endKeyboardInteraction - }; - - this.focusablePath.addInputListener( { - focus: () => { - - // We want the newer interaction to take precedent, so tabbing to the item should interrupt the previous mouse drag (if applicable). - mass.userControlledProperty.value && mass.interruptedEmitter.emit(); - - mass.interruptedEmitter.addListener( endKeyboardInteraction ); - this.focusablePath!.addInputListener( blurListener ); - grabSoundPlayer.play(); - mass.startDrag( mass.matrix.translation ); - } - } ); - const keyboardDragListener = new KeyboardDragListener( { // In model units per second dragSpeed: 3, @@ -146,16 +135,39 @@ tandem: Tandem.OPT_OUT } ); - this.focusablePath.addInputListener( keyboardDragListener ); + // const pt = modelViewTransform.modelToViewPoint( mass.matrix.translation.toVector3().plus( this.tagOffsetProperty.value ).plusXYZ( 0, 0, 0.0001 ) ); + + positionListener(); + this.grabDragInteraction = new GrabDragInteraction( this.focusablePath, keyboardDragListener, { + // grabCueOptions: { // TODO: Position the cue? https://github.com/phetsims/density-buoyancy-common/issues/209 + // center: pt + // }, + onGrab() { + // We want the newer interaction to take precedent, so tabbing to the item should interrupt the previous mouse drag (if applicable). + mass.userControlledProperty.value && mass.interruptedEmitter.emit(); + + mass.interruptedEmitter.addListener( endKeyboardInteraction ); + grabSoundPlayer.play(); + mass.startDrag( mass.matrix.translation ); + }, + onRelease() { + endKeyboardInteraction(); + }, + tandem: Tandem.OPT_OUT + } ); this.disposeEmitter.addListener( () => { + this.grabDragInteraction!.dispose(); keyboardDragListener.dispose(); this.focusablePath && this.focusablePath.dispose(); } ); } - this.mass.transformedEmitter.addListener( this.positionListener ); - this.positionListener(); + this.mass.transformedEmitter.addListener( positionListener ); + + this.disposeEmitter.addListener( () => this.mass.transformedEmitter.removeListener( positionListener ) ); + + positionListener(); } // Override in subclasses to add subclass-specific behavior @@ -172,10 +184,11 @@ public decorate( decorationLayer: MassDecorationLayer ): void { this.massTagNode && decorationLayer.massTagsLayer.addChild( this.massTagNode ); + // this.focusablePath && this.focusablePath.detach(); + // this.focusablePath && decorationLayer.forceDiagramLayer.addChild( this.focusablePath! ); } public override dispose(): void { - this.mass.transformedEmitter.removeListener( this.positionListener ); this.massMesh.dispose(); this.massTagNode && this.massTagNode.dispose(); Index: scenery-phet/js/accessibility/GrabDragInteraction.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/scenery-phet/js/accessibility/GrabDragInteraction.js b/scenery-phet/js/accessibility/GrabDragInteraction.js --- a/scenery-phet/js/accessibility/GrabDragInteraction.js (revision f4d27bbed759716bcda438e49a9a3316142262ef) +++ b/scenery-phet/js/accessibility/GrabDragInteraction.js (date 1723067311596) @@ -715,6 +715,7 @@ * @private */ updateFocusHighlights() { + console.log('udpatin focus highlights') if ( this.grabbable ) { this.node.focusHighlight = this.grabFocusHighlight; this.node.interactiveHighlight = this.grabInteractiveHighlight;
samreid commented 1 month ago
zepumph commented 1 month ago

I thought I may be able to get a lead on the above malalignment by looking at how the p2 body and the Mass.matrix stay in sync. I couldn't really tell if things get off there, but I was pointed to Mass.updateDrag() as a potential lead. Adding these two lines at the end of that function didn't do anything noticeable.

    this.readData();
    this.transformedEmitter.emit();
zepumph commented 1 month ago

There is another TODO to position the grab cue below the mass. I wonder if we will want to create a custom Property to position each at the right spot, like for force/tag/massLabel:

Otherwise, I don't think we can come to a general solution for all mass shapes.

zepumph commented 1 month ago