Create a sound for the wave detector

[samreid]

From meeting notes last week:

KP said it would be good to try out a sound for the meter Let’s try a mapping where the volume of a tone is controlled by the overall amplitude of the wave so that there is no sound when there is no wave, and then have the pitch modulated by the instantaneous value of the wave at the dot on the graph in the meter. SR: Do we just choose a fixed frequency for the dark gray probe and a different frequency for the light gray probe? There should be two different central pitches to support the two probes Who is on the hook to implement? JB: Ashton will provide sounds, Sam will integrate with support from John (Sam - the ForceSoundGenerator in GFLB is likely a good starting point)

[samreid]

From slack, @jbphet said:

If you're inclined to create a common component like "ContinuousPropertySoundGenerator" or something along those lines, that would be great. I've been creating similar things recently for discrete sound generators. If you're on a tight deadline or otherwise disinclined, it'd be okay with me if you copied the one from GFLB and logged an issue about making it a common code component. And assigned the issue to me.

[samreid]

I demoed this patch connecting ContinuousSoundPropertyGenerator today, it was unclear whether we will use this pattern moving forward. It seems we want to try tying the Property to a filter frequency rather than the pitch, and try amplitude changes for the waves. Anyways here is the patch in case it is helpful:

```diff Index: js/common/view/WaveMeterNode.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- js/common/view/WaveMeterNode.js (revision a0a15cc4500ca34f6fec065cebb21ce192894883) +++ js/common/view/WaveMeterNode.js (date 1566324168719) @@ -9,7 +9,9 @@ 'use strict'; // modules + const BooleanProperty = require( 'AXON/BooleanProperty' ); const Color = require( 'SCENERY/util/Color' ); + const ContinuousPropertySoundGenerator = require( 'TAMBO/sound-generators/ContinuousPropertySoundGenerator' ); const DerivedProperty = require( 'AXON/DerivedProperty' ); const DynamicProperty = require( 'AXON/DynamicProperty' ); const DynamicSeries = require( 'GRIDDLE/DynamicSeries' ); @@ -17,6 +19,8 @@ const LabeledScrollingChartNode = require( 'GRIDDLE/LabeledScrollingChartNode' ); const Node = require( 'SCENERY/nodes/Node' ); const NodeProperty = require( 'SCENERY/util/NodeProperty' ); + const Property = require( 'AXON/Property' ); + const Range = require( 'DOT/Range' ); const SceneToggleNode = require( 'WAVE_INTERFERENCE/common/view/SceneToggleNode' ); const ScrollingChartNode = require( 'GRIDDLE/ScrollingChartNode' ); const ShadedRectangle = require( 'SCENERY_PHET/ShadedRectangle' ); @@ -26,10 +30,17 @@ const WaveInterferenceText = require( 'WAVE_INTERFERENCE/common/view/WaveInterferenceText' ); const WaveMeterProbeNode = require( 'WAVE_INTERFERENCE/common/view/WaveMeterProbeNode' ); const WireNode = require( 'SCENERY_PHET/WireNode' ); + const soundManager = require( 'TAMBO/soundManager' ); // strings const timeString = require( 'string!WAVE_INTERFERENCE/time' ); + // sounds + const sineSound = require( 'sound!TAMBO/220hz-saturated-sine-loop.mp3' ); + const stringSound1 = require( 'sound!TAMBO/strings-loop-middle-c-oscilloscope.mp3' ); + const windSound1 = require( 'sound!TAMBO/winds-loop-middle-c-oscilloscope.mp3' ); + const windSound2 = require( 'sound!TAMBO/winds-loop-c3-oscilloscope.mp3' ); + // constants const SERIES_1_COLOR = '#5c5d5f'; // same as in Bending Light const SERIES_2_COLOR = '#ccced0'; // same as in Bending Light @@ -96,7 +107,14 @@ * @param {Property.} connectionProperty * @returns {DynamicSeries} */ - const initializeSeries = ( color, wireColor, dx, dy, connectionProperty ) => { + const initializeSeries = ( color, wireColor, dx, dy, connectionProperty, sound ) => { + const p = new Property( 0 ); + + if ( sound ) { + const continuousPropertySoundGenerator = new ContinuousPropertySoundGenerator( p, sound, new Range( 0.1, 10 ), new BooleanProperty( false ), {} ); + soundManager.addSoundGenerator( continuousPropertySoundGenerator ); + } + const snapToCenter = () => { if ( model.rotationAmountProperty.value !== 0 && model.sceneProperty.value === model.waterScene ) { const point = view.waveAreaNode.center; @@ -161,6 +179,8 @@ if ( scene.lattice.visibleBoundsContains( sampleI, sampleJ ) ) { const value = scene.lattice.getCurrentValue( sampleI, sampleJ ); dynamicSeries.data.push( new Vector2( scene.timeProperty.value, value ) ); + // console.log(value); + p.value = value + 5; } else { dynamicSeries.data.push( new Vector2( scene.timeProperty.value, NaN ) ); @@ -197,8 +217,8 @@ [ leftBottomProperty ], position => position.isFinite() ? position.plusXY( 0, -10 ) : Vector2.ZERO ); - const series1 = initializeSeries( SERIES_1_COLOR, WIRE_1_COLOR, 5, 10, aboveBottomLeft1 ); - const series2 = initializeSeries( SERIES_2_COLOR, WIRE_2_COLOR, 36, 54, aboveBottomLeft2 ); + const series1 = initializeSeries( SERIES_1_COLOR, WIRE_1_COLOR, 5, 10, aboveBottomLeft1, windSound2 ); + const series2 = initializeSeries( SERIES_2_COLOR, WIRE_2_COLOR, 36, 54, aboveBottomLeft2, sineSound ); const verticalAxisTitleNode = new SceneToggleNode( model, ```

[samreid]

From last week's meeting:

@Ashton-Morris said:

What if the sound fades in and out like “woop woop woop” instead of having a continuous “woooooooooooooooooo” like we have now? It would still be frequency matched. We could try turning on sound for just the top half of the wave, with the volume matched.

@emily-phet replied

That sounds worth a try

[samreid]

Here's a patch that implements the behavior described above:

```diff Index: js/common/view/WaveMeterNode.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- js/common/view/WaveMeterNode.js (revision 044bacb0be806de0fb38d6cc3b50583f909883b2) +++ js/common/view/WaveMeterNode.js (date 1566876168225) @@ -9,7 +9,9 @@ 'use strict'; // modules + const BooleanProperty = require( 'AXON/BooleanProperty' ); const Color = require( 'SCENERY/util/Color' ); + const ContinuousPropertySoundGenerator = require( 'TAMBO/sound-generators/ContinuousPropertySoundGenerator' ); const DerivedProperty = require( 'AXON/DerivedProperty' ); const DynamicProperty = require( 'AXON/DynamicProperty' ); const DynamicSeries = require( 'GRIDDLE/DynamicSeries' ); @@ -17,9 +19,13 @@ const LabeledScrollingChartNode = require( 'GRIDDLE/LabeledScrollingChartNode' ); const Node = require( 'SCENERY/nodes/Node' ); const NodeProperty = require( 'SCENERY/util/NodeProperty' ); + const Property = require( 'AXON/Property' ); + const Range = require( 'DOT/Range' ); const SceneToggleNode = require( 'WAVE_INTERFERENCE/common/view/SceneToggleNode' ); const ScrollingChartNode = require( 'GRIDDLE/ScrollingChartNode' ); const ShadedRectangle = require( 'SCENERY_PHET/ShadedRectangle' ); + const soundManager = require( 'TAMBO/soundManager' ); + const Util = require( 'DOT/Util' ); const Vector2 = require( 'DOT/Vector2' ); const Vector2Property = require( 'DOT/Vector2Property' ); const waveInterference = require( 'WAVE_INTERFERENCE/waveInterference' ); @@ -30,6 +36,12 @@ // strings const timeString = require( 'string!WAVE_INTERFERENCE/time' ); + // sounds + const sineSound = require( 'sound!TAMBO/220hz-saturated-sine-loop.mp3' ); + const stringSound1 = require( 'sound!TAMBO/strings-loop-middle-c-oscilloscope.mp3' ); + const windSound1 = require( 'sound!TAMBO/winds-loop-middle-c-oscilloscope.mp3' ); + const windSound2 = require( 'sound!TAMBO/winds-loop-c3-oscilloscope.mp3' ); + // constants const SERIES_1_COLOR = '#5c5d5f'; // same as in Bending Light const SERIES_2_COLOR = '#ccced0'; // same as in Bending Light @@ -96,7 +108,18 @@ * @param {Property.} connectionProperty * @returns {DynamicSeries} */ - const initializeSeries = ( color, wireColor, dx, dy, connectionProperty ) => { + const initializeSeries = ( color, wireColor, dx, dy, connectionProperty, sound ) => { + const p = new Property( 0 ); + + let x = null; + if ( sound ) { + const continuousPropertySoundGenerator = new ContinuousPropertySoundGenerator( p, sound, new Range( 0.1, 5 ), new BooleanProperty( false ), { + // pitchRangeInSemitones: 36, + } ); + soundManager.addSoundGenerator( continuousPropertySoundGenerator ); + x = continuousPropertySoundGenerator; + } + const snapToCenter = () => { if ( model.rotationAmountProperty.value !== 0 && model.sceneProperty.value === model.waterScene ) { const point = view.waveAreaNode.center; @@ -161,8 +184,17 @@ if ( scene.lattice.visibleBoundsContains( sampleI, sampleJ ) ) { const value = scene.lattice.getCurrentValue( sampleI, sampleJ ); dynamicSeries.data.push( new Vector2( scene.timeProperty.value, value ) ); + // console.log( value ); + if ( value > 0 ) { + p.value = value + 5; + const volume = Util.linear( 0, 1.5, 0, 1, value ); + + // TODO: This interferes with the setOutputLevel in the ContinuousPropertySoundGenerator + x.setOutputLevel( volume ); + } } else { + p.value = 0; dynamicSeries.data.push( new Vector2( scene.timeProperty.value, NaN ) ); } } @@ -197,8 +229,8 @@ [ leftBottomProperty ], position => position.isFinite() ? position.plusXY( 0, -10 ) : Vector2.ZERO ); - const series1 = initializeSeries( SERIES_1_COLOR, WIRE_1_COLOR, 5, 10, aboveBottomLeft1 ); - const series2 = initializeSeries( SERIES_2_COLOR, WIRE_2_COLOR, 36, 54, aboveBottomLeft2 ); + const series1 = initializeSeries( SERIES_1_COLOR, WIRE_1_COLOR, 5, 10, aboveBottomLeft1, windSound2 ); + const series2 = initializeSeries( SERIES_2_COLOR, WIRE_2_COLOR, 36, 54, aboveBottomLeft2, sineSound ); const verticalAxisTitleNode = new SceneToggleNode( model, ```

[samreid]

Patch from today's discussion:

```diff Index: wave-interference/js/common/view/WaveMeterNode.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- wave-interference/js/common/view/WaveMeterNode.js (revision 044bacb0be806de0fb38d6cc3b50583f909883b2) +++ wave-interference/js/common/view/WaveMeterNode.js (date 1566936157977) @@ -9,7 +9,9 @@ 'use strict'; // modules + const BooleanProperty = require( 'AXON/BooleanProperty' ); const Color = require( 'SCENERY/util/Color' ); + const ContinuousPropertySoundGenerator = require( 'TAMBO/sound-generators/ContinuousPropertySoundGenerator' ); const DerivedProperty = require( 'AXON/DerivedProperty' ); const DynamicProperty = require( 'AXON/DynamicProperty' ); const DynamicSeries = require( 'GRIDDLE/DynamicSeries' ); @@ -17,9 +19,13 @@ const LabeledScrollingChartNode = require( 'GRIDDLE/LabeledScrollingChartNode' ); const Node = require( 'SCENERY/nodes/Node' ); const NodeProperty = require( 'SCENERY/util/NodeProperty' ); + const Property = require( 'AXON/Property' ); + const Range = require( 'DOT/Range' ); const SceneToggleNode = require( 'WAVE_INTERFERENCE/common/view/SceneToggleNode' ); const ScrollingChartNode = require( 'GRIDDLE/ScrollingChartNode' ); const ShadedRectangle = require( 'SCENERY_PHET/ShadedRectangle' ); + const soundManager = require( 'TAMBO/soundManager' ); + const Util = require( 'DOT/Util' ); const Vector2 = require( 'DOT/Vector2' ); const Vector2Property = require( 'DOT/Vector2Property' ); const waveInterference = require( 'WAVE_INTERFERENCE/waveInterference' ); @@ -30,6 +36,12 @@ // strings const timeString = require( 'string!WAVE_INTERFERENCE/time' ); + // sounds + const sineSound = require( 'sound!TAMBO/220hz-saturated-sine-loop.mp3' ); + const stringSound1 = require( 'sound!TAMBO/strings-loop-middle-c-oscilloscope.mp3' ); + const windSound1 = require( 'sound!TAMBO/winds-loop-middle-c-oscilloscope.mp3' ); + const windSound2 = require( 'sound!TAMBO/winds-loop-c3-oscilloscope.mp3' ); + // constants const SERIES_1_COLOR = '#5c5d5f'; // same as in Bending Light const SERIES_2_COLOR = '#ccced0'; // same as in Bending Light @@ -96,7 +108,18 @@ * @param {Property.} connectionProperty * @returns {DynamicSeries} */ - const initializeSeries = ( color, wireColor, dx, dy, connectionProperty ) => { + const initializeSeries = ( color, wireColor, dx, dy, connectionProperty, sound ) => { + const p = new Property( 0 ); + + let x = null; + if ( sound ) { + const continuousPropertySoundGenerator = new ContinuousPropertySoundGenerator( p, sound, new Range( 0.1, 5 ), new BooleanProperty( false ), { + pitchRangeInSemitones: 60 + } ); + soundManager.addSoundGenerator( continuousPropertySoundGenerator ); + x = continuousPropertySoundGenerator; + } + const snapToCenter = () => { if ( model.rotationAmountProperty.value !== 0 && model.sceneProperty.value === model.waterScene ) { const point = view.waveAreaNode.center; @@ -161,8 +184,20 @@ if ( scene.lattice.visibleBoundsContains( sampleI, sampleJ ) ) { const value = scene.lattice.getCurrentValue( sampleI, sampleJ ); dynamicSeries.data.push( new Vector2( scene.timeProperty.value, value ) ); + // console.log( value ); + if ( true ) { + p.value = value + 5; + const volume = Util.linear( 0, 1.5, 0, 2, Math.abs( value ) ); //TODO: maybe nonlinear mapping here to have more space in the middle + + // TODO: This interferes with the setOutputLevel in the ContinuousPropertySoundGenerator + x.setOutputLevel( volume ); + } + // else { + // x.setOutputLevel( 0 ); + // } } else { + p.value = 0; dynamicSeries.data.push( new Vector2( scene.timeProperty.value, NaN ) ); } } @@ -197,8 +232,8 @@ [ leftBottomProperty ], position => position.isFinite() ? position.plusXY( 0, -10 ) : Vector2.ZERO ); - const series1 = initializeSeries( SERIES_1_COLOR, WIRE_1_COLOR, 5, 10, aboveBottomLeft1 ); - const series2 = initializeSeries( SERIES_2_COLOR, WIRE_2_COLOR, 36, 54, aboveBottomLeft2 ); + const series1 = initializeSeries( SERIES_1_COLOR, WIRE_1_COLOR, 5, 10, aboveBottomLeft1, sineSound ); + const series2 = initializeSeries( SERIES_2_COLOR, WIRE_2_COLOR, 36, 54, aboveBottomLeft2, sineSound ); const verticalAxisTitleNode = new SceneToggleNode( model, Index: tambo/js/sound-generators/ContinuousPropertySoundGenerator.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- tambo/js/sound-generators/ContinuousPropertySoundGenerator.js (revision b7c6af848cca8d473359ed226dc5771788d203dc) +++ tambo/js/sound-generators/ContinuousPropertySoundGenerator.js (date 1566935726538) @@ -33,7 +33,7 @@ loop: true, trimSilence: false, pitchRangeInSemitones: 36, - pitchCenterOffset: 2, + pitchCenterOffset: -10, fadeStartDelay: 0.2, // in seconds, time to wait before starting fade fadeTime: 0.15, // in seconds, duration of fade out delayBeforeStop: 0.1 // in seconds, amount of time from full fade to stop of sound, done to avoid glitches ```

Summary from discussion. We want to sonify the entire waveform, not just the top half. It should sound like wooo waaa wooo waaa so it still has the right frequency. It should have blank space near the 0-crossings so it is a bit thinner. Try it with more pan-flute like sounds.

[samreid]

I moved the proposed changes to a branch

[samreid]

At today's wave interference sonification design meeting, we looked at the SineWave sound effect with a single pitch at the high and another single pitch at the low of the wave. We further refined it by using a triangle wave and using frequencies like setFrequencies(110,150). We also inversed the sin so it would be linear like @Ashton-Morris's demo last week. @kathy-phet said she was looking for more of a harpsichord sound, but we weren't sure how to accomplish that.

[samreid]

It seems we reviewed this sound and did not recommend any changes as part of https://github.com/phetsims/wave-interference/issues/465#issuecomment-583744423. Closing for now, but will reopen if we hear trouble in the Feb 20 meeting.