Utterance to support alerts created from responseCollector

[zepumph]

While brainstorming solutions around https://github.com/phetsims/friction/issues/211, @jessegreenberg and I thought it may be helpful to bring the notion of categories of responses into the utterance-queue library. We came up with a "ResponsePacket" which can be added to the definition in AlertableDef, and that Utterance knows how to support. This involves moving a couple of classes from scenery into utterance-queue. Here is our current patch, I'll take it from here.

```diff Index: scenery/js/accessibility/voicing/responseCollector.js =================================================================== --- scenery/js/accessibility/voicing/responseCollector.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) +++ scenery/js/accessibility/voicing/responseCollector.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) @@ -1,100 +0,0 @@ -// Copyright 2021, University of Colorado Boulder - -/** - * Manages output of responses for the Voicing feature. First, see Voicing.js for a description of what that includes. - * This singleton is responsible for controlling when responses of each category are spoken when speech is - * requested for a Node composed with Voicing. - * - * @author Jesse Greenberg (PhET Interactive Simulations) - */ - -import BooleanProperty from '../../../../axon/js/BooleanProperty.js'; -import merge from '../../../../phet-core/js/merge.js'; -import StringUtils from '../../../../phetcommon/js/util/StringUtils.js'; -import scenery from '../../scenery.js'; -import VoicingResponsePatterns from './VoicingResponsePatterns.js'; - -class ResponseCollector { - constructor() { - - // @public {BooleanProperty} - whether or not component names are read as input lands on various components - this.nameResponsesEnabledProperty = new BooleanProperty( true ); - - // @public {BooleanProperty} - whether or not "Object Responses" are read as interactive components change - this.objectResponsesEnabledProperty = new BooleanProperty( false ); - - // @public {BooleanProperty} - whether or not "Context Responses" are read as inputs receive interaction - this.contextResponsesEnabledProperty = new BooleanProperty( false ); - - // @public {BooleanProperty} - whether or not "Hints" are read to the user in response to certain input - this.hintResponsesEnabledProperty = new BooleanProperty( false ); - } - - /** - * Prepares final output with an object response, a context response, and a hint. Each response - * will only be added to the final string if that response type is included by the user. Rather than using - * unique utterances, we use string interpolation so that the highlight around the abject being spoken - * about stays lit for the entire combination of responses. - * @public - * - * @param {Object} [options] - * @returns {string} - */ - collectResponses( options ) { - - options = merge( { - - // {string|null} - spoken when name responses are enabled - nameResponse: null, - - // {string|null} - spoken when object responses are enabled - objectResponse: null, - - // {string|null} - spoken when context responses are enabled - contextResponse: null, - - // {string|null} - spoken when interaction hints are enabled - hintResponse: null, - - // {boolean} - if true, the nameResponse, objectResponse, contextResponse, and interactionHint will all be spoken - // regardless of the values of the Properties of responseCollector - ignoreProperties: false, - - // {Object} - The collection of string patterns to use when assembling responses based on which - // responses are provided and which responseCollector Properties are true. See VoicingResponsePatterns - // if you do not want to use the default. - responsePatterns: VoicingResponsePatterns.DEFAULT_RESPONSE_PATTERNS - }, options ); - - VoicingResponsePatterns.validatePatternKeys( options.responsePatterns ); - - const usesNames = options.nameResponse && ( this.nameResponsesEnabledProperty.get() || options.ignoreProperties ); - const usesObjectChanges = options.objectResponse && ( this.objectResponsesEnabledProperty.get() || options.ignoreProperties ); - const usesContextChanges = options.contextResponse && ( this.contextResponsesEnabledProperty.get() || options.ignoreProperties ); - const usesInteractionHints = options.hintResponse && ( this.hintResponsesEnabledProperty.get() || options.ignoreProperties ); - const responseKey = VoicingResponsePatterns.createPatternKey( usesNames, usesObjectChanges, usesContextChanges, usesInteractionHints ); - - let finalResponse = ''; - if ( responseKey ) { - - // graceful if the responseKey is empty, but if we formed some key, it should - // be defined in responsePatterns - const patternString = options.responsePatterns[ responseKey ]; - assert && assert( patternString, `no pattern string found for key ${responseKey}` ); - - finalResponse = StringUtils.fillIn( patternString, { - NAME: options.nameResponse, - OBJECT: options.objectResponse, - CONTEXT: options.contextResponse, - HINT: options.hintResponse - } ); - } - - return finalResponse; - } -} - -const responseCollector = new ResponseCollector(); - -scenery.register( 'responseCollector', responseCollector ); -export default responseCollector; Index: scenery/js/accessibility/voicing/VoicingResponsePatterns.js =================================================================== --- scenery/js/accessibility/voicing/VoicingResponsePatterns.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) +++ scenery/js/accessibility/voicing/VoicingResponsePatterns.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) @@ -1,100 +0,0 @@ -// Copyright 2021, University of Colorado Boulder - -/** - * A collection of string patterns that are used with responseCollectorcollectResponses. Responses for Voicing are - * categorized into one of "Name", "Object", "Context", or "Hint" responses. A node that implements voicing may - * have any number of these responses and each of these responses can be enabled/disabled by user preferences - * through the Properties of responseCollector. So we need string patterns that include each combination of response. - * - * Furthermore, you may want to control the order, punctuation, or other content in these patterns, so the default - * cannot be used. VoicingResponsePatterns will have a collections of patterns that may be generally useful. But if - * you need a collection that is not provided here, you can use createResponsePatterns() to create an object based - * on one of the pre-made collections in this file. If you need something totally different, create your own from - * scratch. The object you create must have exactly the keys of DEFAULT_RESPONSE_PATTERNS. - * - * @author Jesse Greenberg (PhET Interactive Simulations) - */ - -import merge from '../../../../phet-core/js/merge.js'; -import scenery from '../../scenery.js'; - -// constants -const NAME_KEY = 'NAME'; -const OBJECT_KEY = 'OBJECT'; -const CONTEXT_KEY = 'CONTEXT'; -const HINT_KEY = 'HINT'; - -const VoicingResponsePatterns = { - - // Default order and punctuation for Voicing responses. - DEFAULT_RESPONSE_PATTERNS: { - nameObjectContextHint: '{{NAME}}, {{OBJECT}}, {{CONTEXT}} {{HINT}}', - nameObjectContext: '{{NAME}}, {{OBJECT}}, {{CONTEXT}}', - nameObjectHint: '{{NAME}}, {{OBJECT}}, {{HINT}}', - nameContextHint: '{{NAME}}, {{CONTEXT}} {{HINT}}', - nameObject: '{{NAME}}, {{OBJECT}}, ', - nameContext: '{{NAME}}, {{CONTEXT}}', - nameHint: '{{NAME}}, {{HINT}}', - name: '{{NAME}}', - objectContextHint: '{{OBJECT}}, {{CONTEXT}} {{HINT}}', - objectContext: '{{OBJECT}}, {{CONTEXT}}', - objectHint: '{{OBJECT}}, {{HINT}}', - contextHint: '{{CONTEXT}} {{HINT}}', - object: '{{OBJECT}} ', - context: '{{CONTEXT}}', - hint: '{{HINT}}' - }, - - /** - * Create an Object containing patterns for responses that are generated by responseCollector.collectResponses. This - * is convenient if you want to use one of the premade collections of patterns above, but have some of the patterns - * slightly modified. - * @public - * - * @param {Object} source - source for merge, probably one of the premade patterns objects in VoicingResponsPatterns - * @param {Object} [options] - Object with keys that you want overridden - * @returns {Object} - */ - createResponsePatterns( source, options ) { - const newPatterns = merge( {}, VoicingResponsePatterns.DEFAULT_RESPONSE_PATTERNS, options ); - VoicingResponsePatterns.validatePatternKeys( newPatterns ); - - return newPatterns; - }, - - /** - * Ensures that keys of the provided pattern will work responseCollector.collectResponses. - * @public - * - * @param {Object} object - */ - validatePatternKeys( object ) { - assert && assert( _.difference( Object.keys( object ), Object.keys( VoicingResponsePatterns.DEFAULT_RESPONSE_PATTERNS ) ).length === 0, - 'keys for the created patterns will not work, they must match DEFAULT_RESPONSE_PATTERNS exactly.' ); - }, - - /** - * Create a key to be used to get a string pattern for a Voicing response. Assumes keys - * are like those listed in DEFAULT_RESPONSE_PATTERNS. - * @public - * - * @param {boolean} includeName - * @param {boolean} includeObject - * @param {boolean} includeContext - * @param {boolean} includeHint - * @returns {string} - string key, could be empty - */ - createPatternKey( includeName, includeObject, includeContext, includeHint ) { - let key = ''; - if ( includeName ) { key = key.concat( NAME_KEY.concat( '_' ) ); } - if ( includeObject ) { key = key.concat( OBJECT_KEY.concat( '_' ) ); } - if ( includeContext ) { key = key.concat( CONTEXT_KEY.concat( '_' ) ); } - if ( includeHint ) { key = key.concat( HINT_KEY.concat( '_' ) ); } - - // convert to camel case and trim any underscores at the end of the string - return _.camelCase( key ); - } -}; - -scenery.register( 'VoicingResponsePatterns', VoicingResponsePatterns ); -export default VoicingResponsePatterns; Index: utterance-queue/js/responseCollector.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- utterance-queue/js/responseCollector.js (date 1629823475849) +++ utterance-queue/js/responseCollector.js (date 1629823475849) @@ -0,0 +1,80 @@ +// Copyright 2021, University of Colorado Boulder + +/** + * Manages output of responses for the Voicing feature. First, see Voicing.js for a description of what that includes. + * This singleton is responsible for controlling when responses of each category are spoken when speech is + * requested for a Node composed with Voicing. + * + * @author Jesse Greenberg (PhET Interactive Simulations) + */ + +import BooleanProperty from '../../axon/js/BooleanProperty.js'; +import merge from '../../phet-core/js/merge.js'; +import StringUtils from '../../phetcommon/js/util/StringUtils.js'; +import scenery from '../../scenery/js/scenery.js'; +import ResponsePacket from './ResponsePacket.js'; +import VoicingResponsePatterns from './VoicingResponsePatterns.js'; + +class ResponseCollector { + constructor() { + + // @public {BooleanProperty} - whether or not component names are read as input lands on various components + this.nameResponsesEnabledProperty = new BooleanProperty( true ); + + // @public {BooleanProperty} - whether or not "Object Responses" are read as interactive components change + this.objectResponsesEnabledProperty = new BooleanProperty( false ); + + // @public {BooleanProperty} - whether or not "Context Responses" are read as inputs receive interaction + this.contextResponsesEnabledProperty = new BooleanProperty( false ); + + // @public {BooleanProperty} - whether or not "Hints" are read to the user in response to certain input + this.hintResponsesEnabledProperty = new BooleanProperty( false ); + } + + /** + * Prepares final output with an object response, a context response, and a hint. Each response + * will only be added to the final string if that response type is included by the user. Rather than using + * unique utterances, we use string interpolation so that the highlight around the abject being spoken + * about stays lit for the entire combination of responses. + * @public + * + * @param {Object} [options] + * @returns {string} + */ + collectResponses( options ) { + + // see ResponsePacket for supported options + options = merge( {}, ResponsePacket.DEFAULT_OPTIONS, options ); + + VoicingResponsePatterns.validatePatternKeys( options.responsePatterns ); + + const usesNames = options.nameResponse && ( this.nameResponsesEnabledProperty.get() || options.ignoreProperties ); + const usesObjectChanges = options.objectResponse && ( this.objectResponsesEnabledProperty.get() || options.ignoreProperties ); + const usesContextChanges = options.contextResponse && ( this.contextResponsesEnabledProperty.get() || options.ignoreProperties ); + const usesInteractionHints = options.hintResponse && ( this.hintResponsesEnabledProperty.get() || options.ignoreProperties ); + const responseKey = VoicingResponsePatterns.createPatternKey( usesNames, usesObjectChanges, usesContextChanges, usesInteractionHints ); + + let finalResponse = ''; + if ( responseKey ) { + + // graceful if the responseKey is empty, but if we formed some key, it should + // be defined in responsePatterns + const patternString = options.responsePatterns[ responseKey ]; + assert && assert( patternString, `no pattern string found for key ${responseKey}` ); + + finalResponse = StringUtils.fillIn( patternString, { + NAME: options.nameResponse, + OBJECT: options.objectResponse, + CONTEXT: options.contextResponse, + HINT: options.hintResponse + } ); + } + + return finalResponse; + } +} + +const responseCollector = new ResponseCollector(); + +scenery.register( 'responseCollector', responseCollector ); +export default responseCollector; Index: scenery-phet/js/accessibility/describers/MovementDescriber.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- scenery-phet/js/accessibility/describers/MovementDescriber.js (revision fa35ddb9654f0cb62f3433358e26c2fe7e68a2c1) +++ scenery-phet/js/accessibility/describers/MovementDescriber.js (date 1629823844703) @@ -13,7 +13,7 @@ import Vector2 from '../../../../dot/js/Vector2.js'; import merge from '../../../../phet-core/js/merge.js'; import ModelViewTransform2 from '../../../../phetcommon/js/view/ModelViewTransform2.js'; -import responseCollector from '../../../../scenery/js/accessibility/voicing/responseCollector.js'; +import responseCollector from '../../../../utterance-queue/js/responseCollector.js'; import voicingUtteranceQueue from '../../../../scenery/js/accessibility/voicing/voicingUtteranceQueue.js'; import AlertableDef from '../../../../utterance-queue/js/AlertableDef.js'; import Utterance from '../../../../utterance-queue/js/Utterance.js'; @@ -150,9 +150,10 @@ phet.joist.sim.utteranceQueue.addToBack( alertable ); // direction changes are an object response; support other alertable, not just Utterance. - voicingUtteranceQueue.addToBack( responseCollector.collectResponses( { - objectResponse: alertable instanceof Utterance ? alertable.alert : alertable - } ) ); + voicingUtteranceQueue.addToBack( alertable ); + // voicingUtteranceQueue.addToBack( responseCollector.collectResponses( { + // objectResponse: alertable instanceof Utterance ? alertable.alert : alertable + // } ) ); this.lastAlertedPosition = this.positionProperty.get(); } Index: utterance-queue/js/Utterance.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- utterance-queue/js/Utterance.js (revision 96fffeb8f200a780a6a2f37642485132a0bd94ff) +++ utterance-queue/js/Utterance.js (date 1629824314870) @@ -22,6 +22,9 @@ import validate from '../../axon/js/validate.js'; import merge from '../../phet-core/js/merge.js'; +import AlertableDef from './AlertableDef.js'; +import responseCollector from './responseCollector.js'; +import ResponsePacket from './ResponsePacket.js'; import utteranceQueueNamespace from './utteranceQueueNamespace.js'; // constants @@ -77,7 +80,7 @@ assert && assert( typeof options.predicate === 'function' ); assert && assert( typeof options.alertStableDelay === 'number' ); assert && assert( typeof options.alertMaximumDelay === 'number' ); - assert && options.alert && assert( typeof options.alert === 'string' || Array.isArray( options.alert ) ); + assert && options.alert && assert( AlertableDef.isAlertableDef( options.alert ) ); assert && options.alert && options.loopAlerts && assert( Array.isArray( options.alert ), 'if loopAlerts is provided, options.alert must be an array' ); @@ -120,6 +123,9 @@ else if ( this.loopAlerts ) { alert = this._alert[ this.numberOfTimesAlerted % this._alert.length ]; } + else if ( this._alert instanceof ResponsePacket ) { + alert = responseCollector.collectResponses( this._alert ); + } else { assert && assert( Array.isArray( this._alert ) ); // sanity check const currentAlertIndex = Math.min( this.numberOfTimesAlerted, this._alert.length - 1 ); Index: scenery/js/accessibility/voicing/Voicing.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- scenery/js/accessibility/voicing/Voicing.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) +++ scenery/js/accessibility/voicing/Voicing.js (date 1629823336751) @@ -27,8 +27,8 @@ import Node from '../../nodes/Node.js'; import scenery from '../../scenery.js'; import MouseHighlighting from './MouseHighlighting.js'; -import responseCollector from './responseCollector.js'; -import VoicingResponsePatterns from './VoicingResponsePatterns.js'; +import responseCollector from '../../../../utterance-queue/js/responseCollector.js'; +import VoicingResponsePatterns from '../../../../utterance-queue/js/VoicingResponsePatterns.js'; import voicingUtteranceQueue from './voicingUtteranceQueue.js'; // options that are supported by Voicing.js. Added to mutator keys so that Voicing properties can be set with mutate. Index: utterance-queue/js/VoicingResponsePatterns.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- utterance-queue/js/VoicingResponsePatterns.js (date 1629823250601) +++ utterance-queue/js/VoicingResponsePatterns.js (date 1629823250601) @@ -0,0 +1,100 @@ +// Copyright 2021, University of Colorado Boulder + +/** + * A collection of string patterns that are used with responseCollectorcollectResponses. Responses for Voicing are + * categorized into one of "Name", "Object", "Context", or "Hint" responses. A node that implements voicing may + * have any number of these responses and each of these responses can be enabled/disabled by user preferences + * through the Properties of responseCollector. So we need string patterns that include each combination of response. + * + * Furthermore, you may want to control the order, punctuation, or other content in these patterns, so the default + * cannot be used. VoicingResponsePatterns will have a collections of patterns that may be generally useful. But if + * you need a collection that is not provided here, you can use createResponsePatterns() to create an object based + * on one of the pre-made collections in this file. If you need something totally different, create your own from + * scratch. The object you create must have exactly the keys of DEFAULT_RESPONSE_PATTERNS. + * + * @author Jesse Greenberg (PhET Interactive Simulations) + */ + +import merge from '../../phet-core/js/merge.js'; +import scenery from '../../scenery/js/scenery.js'; + +// constants +const NAME_KEY = 'NAME'; +const OBJECT_KEY = 'OBJECT'; +const CONTEXT_KEY = 'CONTEXT'; +const HINT_KEY = 'HINT'; + +const VoicingResponsePatterns = { + + // Default order and punctuation for Voicing responses. + DEFAULT_RESPONSE_PATTERNS: { + nameObjectContextHint: '{{NAME}}, {{OBJECT}}, {{CONTEXT}} {{HINT}}', + nameObjectContext: '{{NAME}}, {{OBJECT}}, {{CONTEXT}}', + nameObjectHint: '{{NAME}}, {{OBJECT}}, {{HINT}}', + nameContextHint: '{{NAME}}, {{CONTEXT}} {{HINT}}', + nameObject: '{{NAME}}, {{OBJECT}}, ', + nameContext: '{{NAME}}, {{CONTEXT}}', + nameHint: '{{NAME}}, {{HINT}}', + name: '{{NAME}}', + objectContextHint: '{{OBJECT}}, {{CONTEXT}} {{HINT}}', + objectContext: '{{OBJECT}}, {{CONTEXT}}', + objectHint: '{{OBJECT}}, {{HINT}}', + contextHint: '{{CONTEXT}} {{HINT}}', + object: '{{OBJECT}} ', + context: '{{CONTEXT}}', + hint: '{{HINT}}' + }, + + /** + * Create an Object containing patterns for responses that are generated by responseCollector.collectResponses. This + * is convenient if you want to use one of the premade collections of patterns above, but have some of the patterns + * slightly modified. + * @public + * + * @param {Object} source - source for merge, probably one of the premade patterns objects in VoicingResponsPatterns + * @param {Object} [options] - Object with keys that you want overridden + * @returns {Object} + */ + createResponsePatterns( source, options ) { + const newPatterns = merge( {}, VoicingResponsePatterns.DEFAULT_RESPONSE_PATTERNS, options ); + VoicingResponsePatterns.validatePatternKeys( newPatterns ); + + return newPatterns; + }, + + /** + * Ensures that keys of the provided pattern will work responseCollector.collectResponses. + * @public + * + * @param {Object} object + */ + validatePatternKeys( object ) { + assert && assert( _.difference( Object.keys( object ), Object.keys( VoicingResponsePatterns.DEFAULT_RESPONSE_PATTERNS ) ).length === 0, + 'keys for the created patterns will not work, they must match DEFAULT_RESPONSE_PATTERNS exactly.' ); + }, + + /** + * Create a key to be used to get a string pattern for a Voicing response. Assumes keys + * are like those listed in DEFAULT_RESPONSE_PATTERNS. + * @public + * + * @param {boolean} includeName + * @param {boolean} includeObject + * @param {boolean} includeContext + * @param {boolean} includeHint + * @returns {string} - string key, could be empty + */ + createPatternKey( includeName, includeObject, includeContext, includeHint ) { + let key = ''; + if ( includeName ) { key = key.concat( NAME_KEY.concat( '_' ) ); } + if ( includeObject ) { key = key.concat( OBJECT_KEY.concat( '_' ) ); } + if ( includeContext ) { key = key.concat( CONTEXT_KEY.concat( '_' ) ); } + if ( includeHint ) { key = key.concat( HINT_KEY.concat( '_' ) ); } + + // convert to camel case and trim any underscores at the end of the string + return _.camelCase( key ); + } +}; + +scenery.register( 'VoicingResponsePatterns', VoicingResponsePatterns ); +export default VoicingResponsePatterns; Index: utterance-queue/js/AlertableDef.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- utterance-queue/js/AlertableDef.js (revision 96fffeb8f200a780a6a2f37642485132a0bd94ff) +++ utterance-queue/js/AlertableDef.js (date 1629824200144) @@ -8,6 +8,7 @@ * @author Jesse Greenberg */ +import ResponsePacket from './ResponsePacket.js'; import Utterance from './Utterance.js'; import utteranceQueueNamespace from './utteranceQueueNamespace.js'; @@ -33,6 +34,12 @@ for ( let i = 0; i < alertable.length; i++ ) { isAlertable = isItemAlertable( alertable[ i ] ); if ( !isAlertable ) { break; } + + // TODO: We are going to try to support looping arrays of ResponsePackets, if we cannot, add this back in. + // if ( alertable instanceof ResponsePacket ) { + // isAlertable = false; + // break; + // } } } else { @@ -51,6 +58,7 @@ const isItemAlertable = function( alertable ) { return typeof alertable === 'string' || typeof alertable === 'number' || + alertable instanceof ResponsePacket || alertable instanceof Utterance; }; Index: scenery-phet/js/accessibility/GrabDragInteraction.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- scenery-phet/js/accessibility/GrabDragInteraction.js (revision fa35ddb9654f0cb62f3433358e26c2fe7e68a2c1) +++ scenery-phet/js/accessibility/GrabDragInteraction.js (date 1629823336709) @@ -51,7 +51,7 @@ import FocusHighlightPath from '../../../scenery/js/accessibility/FocusHighlightPath.js'; import KeyboardUtils from '../../../scenery/js/accessibility/KeyboardUtils.js'; import PDOMPeer from '../../../scenery/js/accessibility/pdom/PDOMPeer.js'; -import responseCollector from '../../../scenery/js/accessibility/voicing/responseCollector.js'; +import responseCollector from '../../../utterance-queue/js/responseCollector.js'; import voicingUtteranceQueue from '../../../scenery/js/accessibility/voicing/voicingUtteranceQueue.js'; import animatedPanZoomSingleton from '../../../scenery/js/listeners/animatedPanZoomSingleton.js'; import PressListener from '../../../scenery/js/listeners/PressListener.js'; Index: utterance-queue/js/ResponsePacket.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- utterance-queue/js/ResponsePacket.js (date 1629823574510) +++ utterance-queue/js/ResponsePacket.js (date 1629823574510) @@ -0,0 +1,59 @@ +// Copyright 2019-2021, University of Colorado Boulder + +/** + + * @author Jesse Greenberg + * @author Michael Kauzmann (PhET Interactive Simulations) + */ + +import merge from '../../phet-core/js/merge.js'; +import utteranceQueueNamespace from './utteranceQueueNamespace.js'; +import VoicingResponsePatterns from './VoicingResponsePatterns.js'; + +const DEFAULT_OPTIONS = { + + // {string|null} - spoken when name responses are enabled + nameResponse: null, + + // {string|null} - spoken when object responses are enabled + objectResponse: null, + + // {string|null} - spoken when context responses are enabled + contextResponse: null, + + // {string|null} - spoken when interaction hints are enabled + hintResponse: null, + + // {boolean} - if true, the nameResponse, objectResponse, contextResponse, and interactionHint will all be spoken + // regardless of the values of the Properties of responseCollector + ignoreProperties: false, + + // {Object} - The collection of string patterns to use when assembling responses based on which + // responses are provided and which responseCollector Properties are true. See VoicingResponsePatterns + // if you do not want to use the default. + responsePatterns: VoicingResponsePatterns.DEFAULT_RESPONSE_PATTERNS +}; + +class ResponsePacket { + + /** + * @param {Object} [options] + */ + constructor( options ) { + options = merge( {}, DEFAULT_OPTIONS, options ); + + // @public + this.nameResponse = options.nameResponse; + this.objectResponse = options.objectResponse; + this.contextResponse = options.contextResponse; + this.hintResponse = options.hintResponse; + this.ignoreProperties = options.ignoreProperties; + this.responsePatterns = options.responsePatterns; + } +} + +// @static @public +ResponsePacket.DEFAULT_OPTIONS = DEFAULT_OPTIONS; + +utteranceQueueNamespace.register( 'ResponsePacket', ResponsePacket ); +export default ResponsePacket; \ No newline at end of file Index: gravity-force-lab/js/view/GravityForceLabAlertManager.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- gravity-force-lab/js/view/GravityForceLabAlertManager.js (revision d8a2dc079ae8380255375c69654357084c38507f) +++ gravity-force-lab/js/view/GravityForceLabAlertManager.js (date 1629823336744) @@ -12,7 +12,7 @@ import ISLCObjectEnum from '../../../inverse-square-law-common/js/view/ISLCObjectEnum.js'; import merge from '../../../phet-core/js/merge.js'; import StringUtils from '../../../phetcommon/js/util/StringUtils.js'; -import responseCollector from '../../../scenery/js/accessibility/voicing/responseCollector.js'; +import responseCollector from '../../../utterance-queue/js/responseCollector.js'; import voicingUtteranceQueue from '../../../scenery/js/accessibility/voicing/voicingUtteranceQueue.js'; import ActivationUtterance from '../../../utterance-queue/js/ActivationUtterance.js'; import Utterance from '../../../utterance-queue/js/Utterance.js'; Index: inverse-square-law-common/js/view/ISLCObjectNode.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- inverse-square-law-common/js/view/ISLCObjectNode.js (revision a141159cae65a7ab6f8ac590ace8f64126fcc384) +++ inverse-square-law-common/js/view/ISLCObjectNode.js (date 1629823250614) @@ -18,7 +18,7 @@ import merge from '../../../phet-core/js/merge.js'; import PhetFont from '../../../scenery-phet/js/PhetFont.js'; import Voicing from '../../../scenery/js/accessibility/voicing/Voicing.js'; -import VoicingResponsePatterns from '../../../scenery/js/accessibility/voicing/VoicingResponsePatterns.js'; +import VoicingResponsePatterns from '../../../utterance-queue/js/VoicingResponsePatterns.js'; import DragListener from '../../../scenery/js/listeners/DragListener.js'; import Circle from '../../../scenery/js/nodes/Circle.js'; import Node from '../../../scenery/js/nodes/Node.js'; Index: joist/js/preferences/VoicingPanelSection.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- joist/js/preferences/VoicingPanelSection.js (revision 15c0fb942ccebebb9825bec41be68dab424d6b6e) +++ joist/js/preferences/VoicingPanelSection.js (date 1629823336727) @@ -16,7 +16,7 @@ import PhetFont from '../../../scenery-phet/js/PhetFont.js'; import FocusHighlightFromNode from '../../../scenery/js/accessibility/FocusHighlightFromNode.js'; import VoicingText from '../../../scenery/js/accessibility/voicing/nodes/VoicingText.js'; -import responseCollector from '../../../scenery/js/accessibility/voicing/responseCollector.js'; +import responseCollector from '../../../utterance-queue/js/responseCollector.js'; import Voicing from '../../../scenery/js/accessibility/voicing/Voicing.js'; import voicingManager from '../../../scenery/js/accessibility/voicing/voicingManager.js'; import voicingUtteranceQueue from '../../../scenery/js/accessibility/voicing/voicingUtteranceQueue.js'; Index: friction/js/friction/view/describers/BookMovementDescriber.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- friction/js/friction/view/describers/BookMovementDescriber.js (revision cb15698859693f20ad0a4cd7ec7dd70285799267) +++ friction/js/friction/view/describers/BookMovementDescriber.js (date 1629823932775) @@ -10,6 +10,7 @@ import BorderAlertsDescriber from '../../../../../scenery-phet/js/accessibility/describers/BorderAlertsDescriber.js'; import DirectionEnum from '../../../../../scenery-phet/js/accessibility/describers/DirectionEnum.js'; import MovementDescriber from '../../../../../scenery-phet/js/accessibility/describers/MovementDescriber.js'; +import ResponsePacket from '../../../../../utterance-queue/js/ResponsePacket.js'; import Utterance from '../../../../../utterance-queue/js/Utterance.js'; import friction from '../../../friction.js'; import frictionStrings from '../../../frictionStrings.js'; @@ -64,7 +65,11 @@ // @private - special verbose alert for the first 2 times, then use the default this.bottomUtterance = new Utterance( { - alert: [ downRubFastOrSlowString, downRubFastOrSlowString, DEFAULT_MOVEMENT_DESCRIPTIONS.DOWN ], + alert: new ResponsePacket( { + objectResponse: 'Down', + hintResponse: 'Rub fast or slow' + } ), + // alert: [ downRubFastOrSlowString, downRubFastOrSlowString, DEFAULT_MOVEMENT_DESCRIPTIONS.DOWN ], announcerOptions: { cancelOther: false } Index: scenery/js/accessibility/voicing/ReadingBlock.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- scenery/js/accessibility/voicing/ReadingBlock.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) +++ scenery/js/accessibility/voicing/ReadingBlock.js (date 1629823336717) @@ -27,7 +27,7 @@ import Focus from '../Focus.js'; import ReadingBlockHighlight from './ReadingBlockHighlight.js'; import ReadingBlockUtterance from './ReadingBlockUtterance.js'; -import responseCollector from './responseCollector.js'; +import responseCollector from '../../../../utterance-queue/js/responseCollector.js'; import Voicing from './Voicing.js'; import voicingManager from './voicingManager.js'; Index: scenery/js/main.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- scenery/js/main.js (revision 85a445d0d9afcbaa5491c5e4c54f63d8e59a6e08) +++ scenery/js/main.js (date 1629823336756) @@ -16,7 +16,7 @@ import './accessibility/pdom/PDOMTree.js'; import './accessibility/pdom/PDOMUtils.js'; import './accessibility/voicing/Voicing.js'; -import './accessibility/voicing/responseCollector.js'; +import '../../utterance-queue/js/responseCollector.js'; import './accessibility/voicing/voicingManager.js'; import './accessibility/reader/Cursor.js'; import './accessibility/reader/Reader.js';

[zepumph]

Commit comming soon, but I was able to use this patch to test that this is working as expected. When I changed the response levels with preferences checkboxes, the alerts from the Utterance (i.e. ResponesPacket) were respected.

Index: js/friction/view/describers/BookMovementDescriber.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/friction/view/describers/BookMovementDescriber.js b/js/friction/view/describers/BookMovementDescriber.js
--- a/js/friction/view/describers/BookMovementDescriber.js  (revision eac66e94717f16a7208bc781442da84258f81b21)
+++ b/js/friction/view/describers/BookMovementDescriber.js  (date 1629834080588)
@@ -10,6 +10,7 @@
 import BorderAlertsDescriber from '../../../../../scenery-phet/js/accessibility/describers/BorderAlertsDescriber.js';
 import DirectionEnum from '../../../../../scenery-phet/js/accessibility/describers/DirectionEnum.js';
 import MovementDescriber from '../../../../../scenery-phet/js/accessibility/describers/MovementDescriber.js';
+import ResponsePacket from '../../../../../utterance-queue/js/ResponsePacket.js';
 import Utterance from '../../../../../utterance-queue/js/Utterance.js';
 import friction from '../../../friction.js';
 import frictionStrings from '../../../frictionStrings.js';
@@ -64,7 +65,10 @@

     // @private - special verbose alert for the first 2 times, then use the default
     this.bottomUtterance = new Utterance( {
-      alert: [ downRubFastOrSlowString, downRubFastOrSlowString, DEFAULT_MOVEMENT_DESCRIPTIONS.DOWN ],
+      alert: new ResponsePacket( {
+        objectResponse: 'Down',
+        hintResponse: 'Rub fast or slow'
+      } ),
       announcerOptions: {
         cancelOther: false
       }

So the "rub fast or slow" hint is only given when the hint checkbox is provided.

[zepumph]

All TODOs have been completed above.

We now have support for ResponesPackets in Utterances, and within the arrays Utterance.alert takes. I also took on some house keeping in regards to moving responseCollector et al over to utterance-queue.

@jessegreenberg please give this a review. Within the review please comment on if you feel like I missed anything, or if we aren't completely thinking throught part of this refactor.

Otherwise, I'm excited to use this over in https://github.com/phetsims/friction/issues/222!

[jessegreenberg]

This is looking really great, nice work. I tested it out with a basic example by modifying the hello-world examples in the utterance-queue README, and it is working great.

```html Hello ```

I just changed a couple of namespace references from scenery to utteranceQueueNamespace from moving files to this repo.

Hopefully it is proving useful for friction and other instrumentation!

[zepumph]

Excellent thanks! I'm going to close for now.