Sonification for the compass

[Ashton-Morris]

We have a sound and mapping that we used for GFL:B that we feel will map nicely to the compass tool.

Here is a mockup video.

It is not a perfect representation of it but I did my best in my DAW. The sound is the same sound from GFL:B and I think the mapping in terms of fade in and fade out when interacted with out could apply to this compass. In my mockup I have the possible pitch range mapped at 10 semitones.

I am using a .wav because it loops better and that's what we used in GFL:B its the same file but I have added it to our Sound Files folder.

• felCompassSaturatedSineLoop.wav

[pixelzoom]

Sorry, but I'm not understanding from the mockup video.

What quantity is being mapped to the 10 semitones? And what is the mapping?

Does this sound design work for all 3 types of compasses that appear in the sim? There are 3 types of compass, with 3 very different ways in which the needle aligns itself with the magnetic field.

[Ashton-Morris]

When we talked about it, we did talk about the three types of compasses and how they were different.

This was get get a feel for how it sounded on the first screen only and after using it in the sim discuss further how we wanted to proceed.

I know it will not loop perfectly - but my initial guess at how to map and hear it, would be to map the semitone range (0-10) to the possible 0-360 degree movement of the compass range. If the red part of the compass is pointing north or up it would be 0 semitones, if East 2.5 semitones, if South 5 semitones, if West 7.5 semitones.

The compass one the first screen has more of a physical movement like in real life and we wanted to hear how that sounds.

[pixelzoom]

@Ashton-Morris thanks, that's helpful.

One problem... The mapping you've described in https://github.com/phetsims/faradays-electromagnetic-lab/issues/78#issuecomment-1932746549 does not match the coordinate system used by the sim, which is displayed on the field meter as θ. In this sim, the field angle is at zero degress when the red part of the compass needle is pointing to the right, and positive angle is counterclockwise.

@Ashton-Morris Is there any reason why we should not use the same mapping as the sim, so that an increase in field angle is an increase in pitch? That would be as follows for the direction that the red part of the compass needle is pointing:

• right = 0 degrees = 0 semitones
• up = 90 degress = 2.5 semitones
• left = 180 degress = 5 semitones
• down = 270 degrees = 7.5 semitones

[pixelzoom]

Correction... While positive angles are indeed clockwise, the field meter displays angles in the range [0,180] as positive angles, while angles in the range (180,360) are displayed as a negative angle in the range (-180,0). @arouinfar would prefer not to change this, and proceed with mapping to [0,360).

We also noted that there will be a large jump in pitch with any mapping based on actual angle -- for example when the angle goes from 359 to 0, there will be a transition from highest to lowest pitch. If we want to avoid that, we could do the mapping based on the absolute value of θ displayed on the field meter.

[pixelzoom]

2/13/24 design meeting notes (@arouinfar @Ashton-Morris @emily-phet @pixelzoom)

Let's start with "mirroring" the sound about the x-axis. So mapping |angle| to pitch. We will match the coordinate system of the sim.

[pixelzoom]

We have a sound and mapping that we used for GFL:B that we feel will map nicely to the compass tool.

@Ashton-Morris where can I hear this sound in GFL:B?

[Ashton-Morris]

@pixelzoom In GFL:B The force sound when you move either of the masses is the saturated sine wave that we use often. In this sim it fades in and out nicely and has a nice total pitch range.

[pixelzoom]

Here are some dev versions for discussion at today's design meeting.

Theta (θ) is the field angle at the center of the compass. When θ is zero, the red part of the compass needle points down the positive x-axis (to the right). Positive angles are counterclockwise. Angles are in radians in the model, but I'll use degrees here to make the discussion easier.

1.0.0-dev.12 is the most straightforward approach. It maps θ in the range [0,360) to a pitch range of 10 semitones. The mapping is linear. There is a discontinuity (abrupt change) in pitch at angle=0.

1.0.0-dev.13 is the approach that we identified in our 2/13 design meeting; see https://github.com/phetsims/faradays-electromagnetic-lab/issues/78#issuecomment-1942574595. It reflects θ about the x-axis by normalizing it to the range [0,180], which is then mapped to a pitch range of 10 semitones. So (for example) 175 and 185 will be mapped to the same pitch. The mapping is linear. There are no discontinuties in pitch, and a smooth transition at angles of 0 and 180.

Additional notes:

• This does not track the wobble of the compass on the first 2 screens. In the meantime, test on the Electromagnet or Transformer screen. There are significant challenges with tracking the wobble, which we can discuss. The biggest problem is that the wobble may continue after the user has released the compass, and then we don't know if the compass is moving because the needle is wobbling or because something else is changing the magnetic field.

• As with the field meter, sound plays continuously during the drag cycle of the compass. The fade in and fade out times are 250 ms.

• I reused the implementation that was used for the field meter in #77 -- with a different sound file, different pitch range, and different mapping of Property value to pitch. This reuse is a big potential cost savings, and I'm hopeful that the code can be reused for #80. So significant deviations from what we did for field meter should be weighed against the extra cost of having to implement, test, and maintain different code.

[emily-phet]

I just listened in, and here are some thoughts:

• 1.0.0-dev.13 seemed much more intuitive to interpret to me (on third screen and beyond).
• Not hearing a change as the needle "wiggles" is discrepant, harder to associate the sound with the compass in those situations.
• I had been imagining that the compass sound would play whenever it was in the play area - not just when it was moving. So it's really the change in compass reading we're mapping to. So if it's not moving it goes silent. This makes hearing the wobble very important, that's what we're providing auditory access to.

[pixelzoom]

2/20/24 design meeting notes (@arouinfar @Ashton-Morris @emily-phet @pixelzoom)

Another attempt at more thorough requirements:

• Compass makes sound when it is visible and the needle angle (not field angle) is changing.
• Compass tracks the wobble.
• Sounds fades in and out at 250 ms.
• Pitch range is 10 semitones.
• Mapping is from [0,180] to pitch ("mirrored" about the x-axis)
• Sound file is felCompassSaturatedSineLoop.wav
• Sound stops if the angle has not changed after {N} ms.

@emily-phet mentioned a "squeek" feature that she and @Ashton-Morris would like to investigate. We should clarify this, since it will affect the implementation.

On hold until we decide on next steps.

[pixelzoom]

Notes to self for when work on this resumes...

Since this issue was put on hold today, I did a bit of cleanup that would get in the way of other development tasks. I removed console.log statements, cleaned up TODO items, addressed eslint violations, etc.

Rather than leave CompassSonifier in its current state, where I'd have to listen to it constantly during ongoing implementaton... I did a quick conversion of CompassSonifer to a "model element sonifier" that more closely matches the requirements in https://github.com/phetsims/faradays-electromagnetic-lab/issues/78#issuecomment-1955186961.

I feel like CompassSonifer and FieldMeterSonifer are good (new!) patterns for implementing model and view sonification respectively, so I'll pursue those when work resume. They are built on SoundClip.ts and soundManger.ts from tambo, but could easily use other types of tambo SoundGenerators.

[pixelzoom]

Per @kathy-phet and @emily-phet, we will proceed with publishing FEL 1.0 without sound. So labeling this issues as deferred, to be revisited in a future release.

In https://github.com/phetsims/faradays-electromagnetic-lab/commit/80de7de7952139e6186da72342f74d7e00ea4b42, I disabled the compass sonification and restored the default grab/release sounds. I also added TODO comments that reference this issue, documenting open issues and "next steps".

Related code and sound files are not bundled into the built version (html files) if they are not used, so I have not archived them in GitHub. Related files are:

• FELSonifier.ts
• CompassSonifier.ts
• CompassDragSonifier.ts
• sounds/felCompassSaturatedSineLoop.mp3
• sounds/felCompassSaturatedSineLoop_mp3.ts
• assets/felCompassSaturatedSineLoop.wav