Add sounds for sliders in the Layer Model screen

[jbphet]

We created a custom sound for the slider in the first two screens that controls the greenhouse gas concentration. @Ashton-Morris and I were talking during today's sound design meeting about what to do for the sliders on the Layer Model screen, and he'd like to think about some possibilities for custom sounds for these as well. Here is a screenshot of the sliders as they currently look on this screen:

Assigning to @Ashton-Morris to brainstorm some sounds for these.

[Ashton-Morris]

We reviewed a number of directions in our March 1st Sound Design meeting. Here are the directions we decided to continue in and experiment further with.

Solar Intensity: I showed sound that gets less filtered as the slider goes up in intensity. I'll pick a few sounds that I think might work and also copy the specs of how the filter was implemented to test.

Surface Albedo: I showed a sound that has the "wetness" of a delay (meaning you can hear more delayed sounds or repeats as it is more wet and less when it is less wet) go up as the slider goes up creating a delayed sound. I'll come up with a few sounds that I think fit in the sim and send John a dry version of each and a wet version of each. Once he has them he will implement them with a crossfade between the two.

Infrared Absorbance: This one needs more iteration. I will work on making a sound that sounds like something "passing through" and a sound that sounds like "dropping something into a stew" metaphorically and then once we have those we will crossfade between the too on that slider and test it out.

[jbphet]

In today's meeting we decided to:

• try a unique, photon-esque sound for the base sound used by the middle thresholds in the sliders to see how that sounds
• have Ashton make some variations of those sounds in mockups to help us decide if we want to do that, or just go with the plain sound

The overall goal is to create something that is subtle and non-distracting, but somewhat unique to the situation.

[jbphet]

@emily-phet also suggested trying to vary the sound as the layers are added via the number spinner to match the absorbance slider sound. In other words, when a new layer with high absorbance is added, it sounds different than when a new layer is added with low absorbance.

Also, there should probably be a distinct sound for the case where there the last absorbing layer has been removed, something akin to what we did for the complete removal of all solute in the Molarity sim.

[Ashton-Morris]

I have uploaded a base sound for the layer model sliders to test.

In today's meeting we decided to:

• try a unique, photon-esque sound for the base sound used by the middle thresholds in the slider to see how that sounds
• have Ashton make some variations of those sounds in mockups to help us decide if we want to do that, or just go with the plain sound

The overall goal is to create something that is subtle and non-distracting, but somewhat unique to the situation.

[Ashton-Morris]

Here is a video mock up of how the layer sound would be effected by each sliders interaction. @jbphet @emily-phet

greenhouse-effect-surface-layer-model-slider-sounds-mock-ups.m4v

[emily-phet]

Regarding

greenhouse-effect-surface-layer-model-slider-sounds-mock-ups.m4v

I like it! Having all variations on a single sound puts emphasis on what's different between them, which I like. Since each sliders' sound characteristics are different, I think balancing the sounds across each other and their full span could be tricky, but I'm certain @Ashton-Morris and @jbphet can do it! For example, in the demo, the surface albedo sounds a lot louder than the other sliders to me.

[Ashton-Morris]

Regarding

greenhouse-effect-surface-layer-model-slider-sounds-mock-ups.m4v

I like it! Having all variations on a single sound puts emphasis on what's different between them, which I like. Since each sliders' sound characteristics are different, I think balancing the sounds across each other and their full span could be tricky, but I'm certain @Ashton-Morris and @jbphet can do it! For example, in the demo, the surface albedo sounds a lot louder than the other sliders to me.

That is accurate, I did not spend much time mixing and getting the levels right so some are louder or uneven in the mockup. We can work on that in the sim!

[jbphet]

I added the base sound to the sliders and the number-of-layers control so that we could review the sound in context this week. This was done without the variations on the sounds, such as the filters and echoes. We just reviewed this during this week's sound design meeting, and here are the next steps:

• @Ashton-Morris will "thicken" the sound used for the middle thresholds in the sliders, @jbphet will integrate
• @jbphet will make the boundary sounds used on the sliders be the same basic sound but one octave lower
• @Ashton-Morris will provide new sounds for single, double, and triple layers. The "no layers" sound will be the single layer sound lowered by an octave. @jbphet will integrate.
• @jbphet will start working on filters and echoes to create differences between the slider sounds, based on the mockup linked above

[Ashton-Morris]

I have uploaded these sounds for the layers screen.

• greenhouse-effect-surface-layer-model-absorbing-layers-1.mp3
• greenhouse-effect-surface-layer-model-absorbing-layers-2.mp3
• greenhouse-effect-surface-layer-model-absorbing-layers-3.mp3

And also this sound

• greenhouse-effect-surface-layer-model-base-slider-sound-v2.mp3

[Ashton-Morris]

Adding these sounds that I forgot to for the Infrared Absorbance slider.

How this slider works is that it fades between max volume of greenhouse-effect-surface-layer-model-base-slider-sound-v2-pass-through.mp3

And the max volume of this file greenhouse-effect-surface-layer-model-base-slider-sound-v2-filtered.mp3

While one file is at 100% volume the other is at 0% and on the far ends of the slider.

[jbphet]

I integrated the sounds discussed above and the sound design team reviewed them at our 3/29/2022 meeting. Most everything was approved, but we all felt that the sounds for the layers coming and going rang out for a bit too long, so @Ashton-Morris is going to try shortening these sounds.

[Ashton-Morris]

I have uploaded these sounds which are shorter in length.

• greenhouse-effect-surface-layer-model-absorbing-layers-1-v2.mp3
• greenhouse-effect-surface-layer-model-absorbing-layers-2-v2.mp3
• greenhouse-effect-surface-layer-model-absorbing-layers-3-v2.mp3

[jbphet]

@Ashton-Morris - I've integrated these sounds, and renamed them as follows in the codebase:

• oneAborbingLayer.mp3
• twoAborbingLayers.mp3
• threeAborbingLayers.mp3

[jbphet]

@Ashton-Morris and I reviewed this change at today's sound design meeting, and we both think it sounds pretty good. That's the last item for this issue, so we will check the whole sound design for this screen with @emily-phet next week, and I may run it by the rest of the design team as well.

[jbphet]

We reviewed the layers coming-and-going sound with @emily-phet at this week's sound design meeting and she is good with them. I'll run them by the larger design team this week.

[jbphet]

We didn't get the time to review these sounds in the most recent Greenhouse Effect design meeting, so I said I'd publish a dev version so that @arouinfar and @Matthew-Moore240 could review it asynchronously. I've published a version, it can be found at https://phet-dev.colorado.edu/html/greenhouse-effect/1.1.0-dev.8/phet/greenhouse-effect_en_phet.html.

@arouinfar and @Matthew-Moore240 - We're looking for feedback on the sounds associated with the two panels on the Layer Model screen, which basically means the sliders and the spinner for the number of layers. Please have a look and a listen and let us know if you're good with these or if you feel they need further refinement and, if so, how.

@arouinfar - Feel free to share the link with Mark N if you're so inclined. I believe that was mentioned in the meeting.

Tagging @kathy-phet in case she wants to take a look.

[arouinfar]

Thanks @jbphet! I passed on the link to Mark and @Matthew-Moore240 and I will review in our next meeting.

[Matthew-Moore240]

Notes for the sounds:

1. The number spinner sound effect is very good
2. All the sliders seem to cut themselves off
3. The surface albedo sound effect has an echo or reverb effect to it that makes it sound like it crackles a little bit
4. The number spinner will still play a sound effect even if you are at 0 - 3 and are still trying to move up or down when at 0 - 3

[jbphet]

Notes from the review meeting with the rest of the design team:

• [x] apply the filter to the end sounds on the Solar Intensity slider
• [x] try reverb instead of delay for the albedo slider, and apply it to the end sounds as well
• [x] the sounds in the upper range of the IR absorbance slider sound a bit "clipped". What about essentially doing the inverse of the Solar Intensity slider? We could maybe user a slightly different core sound, or maybe not. We would filter the "end" sounds as described above.
• [x] on the number-of-layers number picker, pressing up and down arrows on the keyboard should not continue to produce sounds, as they do on v1.1.0-dev.8

[jbphet]

With respect to the item in the comment above that says:

on the number-of-layers number picker, pressing up and down arrows on the keyboard should not continue to produce sounds, as they do on v1.1.0-dev.8

@emily-phet requested that I ask about this on the a11y-dev Slack channel. I did, and it resulted in a discussion and subsequent design meeting that just concluded. In that meeting, we decided to implement the behavior where attempts to go beyond the min/max values using keyboard nav would result in no sound, but also that some further investigation should be done to make sure that this is what we want for this case and for all similar controls, such as sliders. I'll implement the 'no sound' behavior for now, and will log a separate issue for follow-up.

[Ashton-Morris]

We talked about trying a few impulse responses to add reverb to the albedo slider. So I am adding a few in this folder to test out.

• IR_Response_Large_Room.wav
• IR_Response_Small_Room.wav

• [x] try reverb instead of delay for the albedo slider, and apply it to the end sounds as well

[jbphet]

We discussed the comment above that says,

What about essentially doing the inverse of the Solar Intensity slider? We could maybe user a slightly different core sound, or maybe not. We would filter the "end" sounds as described above.

I'll try just doing the inverse to start with, then will review with the sound design team and we can decide if we want a different core sound.

[jbphet]

I tried the large room IR above, and the effect is a little too subtle. @Ashton-Morris is going to see if he can find a longer impulse response that we could try instead.

[jbphet]

I made the sound generator for IR absorbance act basically like the inverse of the solar intensity generator, but it felt a little off to me since one has to to with the visible light and the other IR light. I tried lowering the base sound by one musical step, and thought it sounded better. We should review this in our next sound design meeting.

[Ashton-Morris]

I added a longer IR response and assured that it was the appropriate volume/gain.

• IR_Response_Large_Hall.mp3
• IR_Response_Large_Hall.wav

[jbphet]

I tried out IR_Response_Large_Hall.mp3 and it's better, but now it feels like a bit too much to me, even when I turn the gain way down on the reverb effect. @emily-phet thought so too when we checked it out in today's sound design meeting. She described it as being a bit "ocean-y". So, can we get something in between the large room and the large hall, like a small or medium hall?

If you want to hear it in context, it's on master at the moment.

[Ashton-Morris]

Adding these files now. Hopefully one of these will fit nicely.

• IR_Response_Medium_Hall.mp3
• IR_Response_Medium_Hall.wav
• IR_Response_Small_Hall.mp3
• IR_Response_Small_Hall.wav

[jbphet]

Note to self:

• [ ] Before closing, make sure all the sounds and impulse responses that were added but don't end up being used get removed from greenhouse-effect and tambo.

[jbphet]

This is done. In the end, the reverb effect was achieved using an existing reverb sound that we already had. Closing.

[jbphet]

For future reference, the length of the tail on the reverb that gave an audible but not over-the-top effect was about 0.9 seconds when viewed in Audacity.