Sonification for the field meter

[Ashton-Morris]

I have a loop that we used in John Travoltage that we feel fits the field meter tool. I have created a mockup video of how it looks and sounds.

Here is the video

In my mockup video I have a 12 semitone range for how low or high the pitch/playback speed changes. Which we want mapped to the top number in the field meter "B"

If you need more info on how it could be put in, I would suggest that you start by looking at how the Force sound is working in Gravity Force Lab: Basics Because it has a loop and a fade in and fade out when you interact with it, and it also changes in pitch.

Here are the sounds. All sounds for this sim will be stored in that folder.

• felFieldMeterLoop.mp3
• felFieldMeterLoop.wav

[pixelzoom]

In my mockup video I have a 12 semitone range for how low or high the pitch/playback speed changes. Which we want mapped to the top number in the field meter "B"

How do you want B mapped to pitch? If it's a linear mapping, you will hear little or no difference in pitch outside the magnet.

B in the field meter is the field magnitude, gauss (G). It's range is [0.01,300] G, 5 orders of magnitude. In the center of the bar magnet, it matches the "Strength" control setting. With "Strength" set to 300G, just outside the bar magnet B immediately drops to 5G at the top/bottom edges, 100G at the left/right ends. And it drops off very quickly with the distance from the magnet.

[Ashton-Morris]

I do not know exactly if its linear mapping or what it would be called. I thought that it could be done similar to the video.

0 G = -12 semitones and 300 G = the initial playback rate.

[pixelzoom]

In the mockup video, it sounds like you're mapping distance (of the field meter from the magnet) to pitch, not the value of B at the field meter's location. Distance is indeed a linear mapping, but that will not work for B. You will hear no difference in pitch anywhere outside the magnet.

I also discussed with @arouinfar, and she suggested that the mapping of B to pitch should be logarithmic, so I'll start there.

[pixelzoom]

Notes to self about implementation in GFLB ...

The force sound is saturatedSineLoopTrimmed.wav, and I used that to track down the relevant code in GFLBScreenView.js:

    // @private - sound generation for the force sound
    this.forceSoundGenerator = new ContinuousPropertySoundGenerator(
      model.forceProperty,
      saturatedSineLoopTrimmed_wav,
      new Range( model.getMinForce(), model.getMaxForce() ),
      {
        initialOutputLevel: 0.2,
        playbackRateCenterOffset: 0.122, // this is about 2 semitone, and was necessary to match original sound design
        resetInProgressProperty: model.resetInProgressProperty,
        trimSilence: false // a very precise sound file is used, so make sure it doesn't get changed
      }
    );
    soundManager.addSoundGenerator( this.forceSoundGenerator );

ContinuousPropertySoundGenerator looks like it only supported linear mappings, so I can't just reuse this code directly. Maybe pass in a DerivedProperty for the field magnitude's exponent?

It needs to be reset when the Reset All button is pressed:

    const resetAllButton = new ResetAllButton( {
      listener: () => {
...
        this.forceSoundGenerator.reset();
      },

It also needs to be stepped, which will add a new level of complexity to FEL. Nothing in the view currently needs to be stepped, and the view has no step method.

  step( dt ) {
    this.forceSoundGenerator.step( dt );
...
  }
``

[pixelzoom]

2/13/24 design meeting notes (@arouinfar @Ashton-Morris @emily-phet @pixelzoom)

Let's start with a non-linear mapping, similar to FieldNode strengthToAlpha that was done for visualization of the magnetic field. The mapping is to pitch, no change in output level. Output level is for sound fade in and out.

[pixelzoom]

The resetInProgressProperty option to ContinuousPropertySoundGenerator was a surprise, presumably added so that we don't get spurious sounds when resetAllButton is pressed. A better name would be updateEnabledProperty. And it's handled poorly in all sims that use it, usually via a model Property that is toggled between true/false in model.reset. Then that model Property needs to be passed through multiple constructors until it gets to where the ContinuousPropertySoundGenerator instance is created. What a pain.

@zepumph suggested doing something similar to PhET-iO's isSettingPhetioStateProperty.ts. So I'll probably add something like this to FEL:

// isResettingAllProperty.ts
const isResettingAllProperty = new TinyProperty( false );
faradaysElectromagneticLab.register( 'isResettingAllProperty', isResettingAllProperty );
export default isResettingAllProperty;

Then wire it into FELScreenView:

    this.resetAllButton = new ResetAllButton( {
      listener: () => {
       isResettingAllProperty.value = true;
        this.interruptSubtreeInput(); // cancel interactions that may be in progress
        options.resetAll();
       isResettingAllProperty.value = false;
      },
      tandem: options.tandem.createTandem( 'resetAllButton' )
    } );

EDIT: Note that @zepumph created https://github.com/phetsims/tambo/issues/186 for addressing this more generally.

[pixelzoom]

@arouinfar @Ashton-Morris @emily-phet: I have a prototype ready for review.

Please test drive in phettest or in 1.0.0-dev.11, with both pointer (mouse, touch) and keyboard input. We had not discussed keyboard input yet, and the video mockups do not demonstrate keyboard input. So what you'll hear is what @arouinfar and I came up with for the prototype.

The mapping between B (magnitude) and playback rate uses the same algorithm that maps B to opacity (the field visualization as a grid of compass needles). I'm pleased that this seems to work well for both visuals and sound.

Current settings that can be tweaked:

• Playback rate range = [1,2], which is one octave (12 semitones)
• Nominal output level = 0.7, where the valid range is [0,1]
• Fade in time = 250 ms
• Fade out time = 500 ms

If what I've done here is in the right ballpark, then next steps will be to generalize the implementation, and apply it to the compass (#78) and electromagnet current (#80).

[emily-phet]

For the sound range of B value = 0-40 G, the sound is nice, and the pitch change is nicely perceptible. However, in the 40-300 G magnetic field range, the sound is nice...but I'm unable to detect a noticeable change in sound. This is most apparent with the Field Meter on the left and right sides (close to the magnet, but not on the magnet).

For this interaction, we need a "just noticeable difference" of 5-10 G throughout the entire 0-300 G magnetic field range. This would ensure learners can discern pitch changes over field differences of 5-10 G. Otherwise the sound will likely feel discrepant from changes in the value of the strength of the magnetic field.

Regarding other sound factors:

• Overall, I like the sound for this purpose. This kind of continuous, relatively full sound usually comes with lots of problems...but I can tell already there's been effort to avoid those issues. It's a little ghost-y but not too much, and at higher values does feel reminiscent of a charged field. Nice @Ashton-Morris!
• I like the effect with the DC power source in later screens. With low amplitude, it reminds me of a heartbeat.
• The sound needs to not play when B = 0.00 G on the meter. Currently still can be heard....presumably it's not actually at 0.
• The current fade-in rate feels good to me during use.
• The fade out rate could be faster, can we try the same as the fade in rate (250ms)? If it's too short it might sound clipped, but right now it feels a tad too long.
• The interaction and sound feel nice with both pointer and keyboard inputs. Although I'm a proponent of the two-step grab for four-way draggable objects with alternative input methods, I expect this approach has been previously considered and passed on for this sim. If that's the case, no need to revisit.

[pixelzoom]

@emily-phet said:

... For this interaction, we need a "just noticeable difference" of 5-10 G throughout the entire 0-300 G magnetic field range. This would ensure learners can discern pitch changes over field differences of 5-10 G. Otherwise the sound will likely feel discrepant from changes in the value of the strength of the magnetic field.

@arouinfar and I experimented with different mappings of B to pitch. We have a couple of choices to demo at design meeting, include what we've called a "piecewise linear" mapping.

In addition to the goals that @emily-phet stated, these points feel pedagogically important:

• Big discontinuity in pitch when entering/leaving the magnet from the top/bottom edges.
• No discontinuity in pitch when entering/leaving the magnet at the poles.
• A noticeable difference in pitch in a roughly circular area near the poles.
• Fast drop-off as distance from the magnet increases, with little/no discernible pitch difference in places that are "far" from the magnet.

The sound needs to not play when B = 0.00 G on the meter. Currently still can be heard....presumably it's not actually at 0.

See #84. We are now displaying small non-zero values as "< 0.01 G" instead of "0.00 G".

The fade out rate could be faster, can we try the same as the fade in rate (250ms)? If it's too short it might sound clipped, but right now it feels a tad too long.

Done.

[Ashton-Morris]

I think it sounds pretty good as it is in currently with the pitch mapping and the sound source.

I am curious if adding a slight low pass filter effect to the sound field meter would add to it not being too in your face. If we did this I think the cuttoff frequency would be mapped pretty similarly as the pitch is and would get more filtered/muffled as the field meter got lower in number.

Also I am not entirely sure but we should ask EM and AR in out meeting today: But I thought that they intended for the field meter to still sound if it was checked but you were moving the magnet around the area while the field meter was sitting out.

[pixelzoom]

2/20/24 design meeting notes (@arouinfar @Ashton-Morris @emily-phet @pixelzoom)

@Ashton-Morris said above:

Also I am not entirely sure but we should ask EM and AR in out meeting today: But I thought that they intended for the field meter to still sound if it was checked but you were moving the magnet around the area while the field meter was sitting out.

We confirmed that the field meter should only play sound while interaction with it is occurring.

We reviewed https://github.com/phetsims/faradays-electromagnetic-lab/issues/77.

I will published a dev version that uses the new "piecewise linear" mapping (and explain it better).

On hold until we decide on next steps.

[pixelzoom]

I will published a dev version that uses the new "piecewise linear" mapping (and explain it better).

See https://phet-dev.colorado.edu/html/faradays-electromagnetic-lab/1.0.0-dev.14/phet/faradays-electromagnetic-lab_all_phet.html

Here's what @arouinfar and I called the "piecewise linear" mapping:

• The "cutoff" P divides the range into 2 pieces
• Default for P = 20G (this can be changed with the ?piecewiseCutoff query parameter)
• For [0, P] G, map linearly to semitones [1,6].
• For [P,300] G, map linearly to semitones [6,12]

This mapping seems to tick all the "pedagocially important" bullets that we listed in https://github.com/phetsims/faradays-electromagnetic-lab/issues/77#issuecomment-1954674745.

[pixelzoom]

Per @kathy-phet and @emily-phet, we will proceed with publishing FEL 1.0 without sound. So labeling this issues as deferred, to be revisited in a future release.

In https://github.com/phetsims/faradays-electromagnetic-lab/commit/22ce21ccccfe67d52f39770e2405d9b0e04176a0, I disabled the field meter sonification and restored the default grab/release sounds. I also added TODO comments that reference this issue, documenting open issues and "next steps".

Related code and sound files are not bundled into the built version (html files) if they are not used, so I have not archived them in GitHub. Related files are:

• FELSonifier.ts
• FieldMeterDragSonifier.ts
• sounds/felFieldMeterLoop.mp3
• sounds/felFieldMeterLoop_mp3.ts
• assets/felFieldMeterLoop.wav