Implement sound design - Githubissues

jbphet commented 4 years ago

@Ashton-Morris, @emily-phet, and I (@jbphet) did some brainstorming on the sound design for the proportion sim and thought we should write some of these things down.

Use two sound generators that are producing the same note when in correct proportion, and when the proportions aren't correct they would produce different notes and thus beat frequencies.
Same idea - two sound generators - but produce the notes periodically so that it doesn't drive people crazy with continuous sounds
Two rhythmic beats that are distinct and line up when the proportions are correct - in a fun way, using double time, triplets, etc, but get out of sync if the proportions are incorrect. They'd speed up or slow down the overall tempo as the fundamental sizes change.

zepumph commented 4 years ago

Decisions from today's meeting:

[x] We prefer the marimba sound to the pizz sounds. That said it leans a bit towards a "casino sound." @Ashton-Morris will work on some fine tuning to the marimba sound so that it is more like a "bing" and less like a "ding."
[x] As for the frequency of the staccato notes (which @jbphet pointed out is actually a period of time between sounds), we will update the default to be 120 and change the name to reflect "period" instead of "frequency."
[x] update "moving proportion" sound tolerance (perhaps half of the tolerance)
[x] Fine tuning step to make sure that you could change directions of velocities without stopping the choir sound.
[ ] "jarring ending of sounds" audit needed (superset of the above bullet)
[ ] fading in/out of moving in proportion sound.
[x] @jbphet mentioned that you can move through and past a ratio success sound without hearing the success. Perhaps it would be nice to always have a success sound when there has been a success, even though the ratio success didn't linger. Perhaps the important thing is just to make sure that "some sound" plays in this case. We want to discuss this further in a future design meeting.
[x] For staccato ratio success sounds, we would like to hear c4-ding-ring-out.mp3 and glock-marimba-c-maj-7-arp.wav in the sim. These both have long tails, and it would potentially be nice to play the tail for as long as the ratio stays in proportion (sorta like a continuous-feel), but then to dampen it when moving out of proportion so that it doesn't linger too long. (implementation notes: use setOutputLevel's second arg, timeConstant, to fade that more).
[x] We feel good about removing all of the current staccato success sounds, replacing them for the above checkbox.
[x] (see https://github.com/phetsims/tambo/issues/110) Move SoundClipChord.js into tambo

zepumph commented 4 years ago

@jbphet mentioned that you can move through and past a ratio success sound without hearing the success. Perhaps it would be nice to always have a success sound when there has been a success, even though the ratio success didn't linger. Perhaps the important thing is just to make sure that "some sound" plays in this case. We want to discuss this further in a future design meeting.

I reworked the logic for the staccato success/in proportion sound, and I think this is much better. Before the algorithm only tried to play the success sound based on the same loop delay as for the normal marimba approaching sound. This meant that the success sound was not eligible for playing every step. Which it should be, so I pulled the success sound out of the logic that sees if enough time has passed to warrant a new approach sound. If that doesn't make sense, then try it out and see if it is better. I'm going to check this box off, but please uncheck if there is more to discuss.

Ashton-Morris commented 4 years ago

Here are some Marimba variations to try out

marimba-variations-003.mp3
marimba-variations-003.wav
marimba-variations-002.mp3
marimba-variations-002.wav
marimba-variations-001.mp3
marimba-variations-001.wav
marimba-variations-004.mp3
marimba-variations-004.wav
marimba-variations-000.mp3
marimba-variations-000.wav

zepumph commented 4 years ago

From today's sound design meeting:

[x] We want to see the above sounds as options in the sim. We don't need to see option (1) or (4)
[x] We want to try changing the pitch based on fitness. Perhaps trying a pentatonic scale, or maybe whole tone.
[x] @Ashton-Morris will experiment with some non-tonal sounds (since we ditched the original tremolo-to-third approach).
[x] (see https://github.com/phetsims/ratio-and-proportion/issues/63) @jbphet mentioned that it is potentially possible for web audio to give you a notification when setOutputLevel is finished with its time-delayed level mutation. I may want to try to use that for the staccato success sound. I will create an issue in tambo to investigate if it is needed.
[x] If I have way too much time to spend, then I could try changing the c major sin sound to layering in chords instead of volume, based on fitness.

zepumph commented 4 years ago

Added the variations on marimba above. Note I also made the staccato sound quieter (along with the success sound).

emily-phet commented 4 years ago

@zepumph I'd love to hear the staccato sounds with the pitch change, but at the moment (without that in place), i like the last option (pluck against drywall) the best. It has just enough of a natural feel to not sound like it could be an error indicator (which the ploink and thoink have a little of).

Also a thought related to pitch change - maybe we're trying to hard to get everything to align to "fitness". It might be worth trying a simpler pitch change that is based just on the y-location, so same pitch change as you go up/down regardless of fitness.

Ashton-Morris commented 4 years ago

Here are some non pitched sounds to try. They are more like button or UI sounds, some of which are from our previous UI tests.

non-melodic-indicator-1.wav non-melodic-indicator-1-002.mp3 non-melodic-indicator-1-002.wav non-melodic-indicator-1-001.mp3 non-melodic-indicator-1-001.wav non-melodic-indicator-1.mp3

zepumph commented 4 years ago

From sound design meeting today:

[x] implement the variation in TODOs in StaccatoFrequencySoundGenerator
[x] Include a way to have marimba tone with frequency modulation but NO pitch modulation, for interview comparisons
[x] Discuss with @jbphet about how to handle added code to MultiClip.js

zepumph commented 4 years ago

Alright, I think that I got to everything we discussed today during sound design meeting. Here is a version with the "random mix" of sounds for staccato by default, and also an option to use the marimba. When on the marimba sound, the pitch won't modulate.

https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.27/phet/ratio-and-proportion_en_phet.html

zepumph commented 4 years ago

Next design meeting it would be nice to take a quick look at the layering done in the c major sin waves. I doubt we will change our strategy now, but it sounds pretty nice no matter.

zepumph commented 4 years ago

We also never discussed @Ashton-Morris's non-melodic sounds. Otherwise, I have done all for this issue that I can right now. Unassigning.

emily-phet commented 4 years ago

To discuss:

[ ] Ashton's non-melodic sounds.
[ ] @zepumph layered C major sin waves (if not for this sim, would be interesting to consider mentioning this for Waves Intro graph challenge.
[ ] A version of the sim without tempo being used - so just movement results in single sound feedback. @zepumph - can you make a version where there is no repetition of sounds as you're interacting? I like the random mix, but the repetition I think makes it harder to interpret what's happening. I think reviewing in advance some clips from the latest interviews (seeing how others interact, how much of their movement is faster vs slower, etc.) will help us interpret if this is an issue or not.
[ ] Simple success sounds. Right now I like the "Random Mix" with "Single High Note", but if this is the general path forward would need some tweaking to both sets of sounds.
[ ] Probably not enough time or final decisions on staccato sound and success sound... but will eventually need to revisit the simultaneous movement success sound.

zepumph commented 4 years ago

Discussion from design meeting today:

It would be good to make some decisions, to help us focus on a prototype path
Perhaps we can experiment with a DMZ of sorts, in which the staccato success not only has to leave its range, but also pass through a DMZ threshold surrounding it in order to then be able to trigger the staccato success again.
@BLFiedler mentioned that it is strange that the jump implemented in https://github.com/phetsims/ratio-and-proportion/issues/69 doesn't match the staccato success sound. We could try out making the staccato success match the threshold for the color, and then we could have a DMZ that adds up to the current breadth of the staccato success threshold.
@emily-phet had a great idea to experiment with taking the frequency increasing piece out of the sound. This new sound would just play a sound every time fitness changes enough. The key here is to make sure that a sound can only play every X (time unit). This will help jumping to the extremes. We can experiment with this as the default sound, and give it a change for a week. We may just choose to revert it next week. Something like this, but it doesn't have the minimum wait time in place yet:

```diff Index: js/common/view/sound/StaccatoFrequencySoundGenerator.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- js/common/view/sound/StaccatoFrequencySoundGenerator.js (revision b0ed1308860a16ca20dbf3306d12c269a531ee6f) +++ js/common/view/sound/StaccatoFrequencySoundGenerator.js (date 1592948920513) @@ -21,11 +21,11 @@ import designingProperties from '../../designingProperties.js'; import RatioAndProportionQueryParameters from '../../RatioAndProportionQueryParameters.js'; -const SUCCESS_OUTPUT_LEVEL = 1; +const SUCCESS_OUTPUT_LEVEL = .8; const INITIAL_PLAYBACK_RATE = 1; // Playback rate will be between 1 and a major third above -const getPlaybackRate = fitness => fitness * ( Math.pow( 2, 4 / 12 ) - INITIAL_PLAYBACK_RATE ) + INITIAL_PLAYBACK_RATE; +const getPlaybackRate = fitness => fitness * ( Math.pow( 2, 7 / 12 ) - INITIAL_PLAYBACK_RATE ) + INITIAL_PLAYBACK_RATE; const BRIGHT_MARIMBA_VALUE = 0; @@ -92,7 +92,13 @@ // @private this.fitnessChanged = true; - fitnessProperty.lazyLink( () => {this.fitnessChanged = true;} ); + let previousFitness = 1; + fitnessProperty.lazyLink( newValue => { + if ( Math.abs( previousFitness - newValue ) > .2 ) { + this.fitnessChanged = true; + previousFitness = newValue; + } + } ); } /** @@ -134,7 +140,7 @@ this.successSoundClip.play(); this.playedSuccessYet = true; } - else if ( this.timeSinceLastPlay > this.timeLinearFunction( newFitness ) && !isInRatio ) { + else if ( true && !isInRatio ) { // Don't modulate pitch for marimba sound if ( designingProperties.staccatoSoundSelectorProperty.value !== BRIGHT_MARIMBA_VALUE ) { ```

For the staccato success sound. We talked about prototyping a couple of different sounds. Our main complaints about the current two options is that they are very sharp. A Rhodes piano chord/note/arpeggio seemed good to try. We also want to prototype on some less-European sounds, to see if we like those. @Ashton-Morris will go for it.

zepumph commented 4 years ago

I added a time-consistent staccato sound option above, and made it default. There are two query parameters to control it. The first has a use in the frequency modulated version also (staccatoMinRepeatTime), and in this mode it is the min time between to sounds, regardless of if the fitness changed enough to trigger that sound. The second is what the fitness change threshold is (fitnessChangeThreshold).

Here is an example of a very ugly version of these query parameters, they play to often back to back, and with too little fitness change: http://phettest.colorado.edu/ratio-and-proportion/ratio-and-proportion_en.html?ea&brand=phet&staccatoMinRepeatTime=10&fitnessChangeThreshold=.05

The defaults are http://phettest.colorado.edu/ratio-and-proportion/ratio-and-proportion_en.html?ea&brand=phet&staccatoMinRepeatTime=120&fitnessChangeThreshold=.2 (same as omitting the query parameters)

Please see https://github.com/phetsims/ratio-and-proportion/blob/6cd3eb6c6840c1571244327aa0765e76b1b464b9/js/common/RatioAndProportionQueryParameters.js#L27-L44 for more explanation on the query parameters.

[ ] Next, hopefully later this afternoon, I will work on a DMZ pattern for the success sound, and maybe pull that into its own sound generator (since it isn't really a staccato feature).

Ashton-Morris commented 4 years ago

Here are some new success sounds.

success-indicator-elec-piano-2w.mp3
success-indicator-elec-piano-2w.wav
success-indicator-elec-piano.mp3
success-indicator-elec-piano.wav
success-indicator-eastern-plucked-instrument-2.mp3
success-indicator-eastern-plucked-instrument-2.wav
success-indicator-eastern-plucked-instrument.mp3
success-indicator-eastern-plucked-instrument.wav

zepumph commented 4 years ago

From today's sound meeting:

[x] Remove the sound mode added last week that didn't increase frequency as you go closer to in Proportion.
[ ] ~Discretize the tones to semitones.~ @Ashton-Morris will give me sounds to implement instead.
[ ] ~Investigate https://github.com/phetsims/tambo/issues/114 as an alternative to needing many different notes of the same sound from @Ashton-Morris to get each to have the same length (since we modulate playback rate to change pitch).~ We decided there was no time right now for this. Instead @Ashton-Morris will eventually make sounds for each tone and sound clip combo.

zepumph commented 4 years ago

Now I am just waiting on sounds from @Ashton-Morris. Unassigning.

Ashton-Morris commented 4 years ago

All right @zepumph here are these files

marimba-variation-v2-c.mp3
marimba-variation-v2-c-001.mp3
marimba-variation-v2-c-002.mp3
marimba-variation-v2-c-sharp-001.mp3
marimba-variation-v2-c-sharp-002.mp3
marimba-variation-v2-c-sharp.mp3
marimba-variation-v2-d.mp3
marimba-variation-v2-d-001.mp3
marimba-variation-v2-d-002.mp3
marimba-variation-v2-d-sharp-001.mp3
marimba-variation-v2-d-sharp-002.mp3
marimba-variation-v2-d-sharp.mp3
marimba-variation-v2-e-001.mp3
marimba-variation-v2-e-002.mp3
marimba-variation-v2-e.mp3
marimba-variation-v2-f-002.mp3
marimba-variation-v2-f.mp3
marimba-variation-v2-f-001.mp3
marimba-variation-v2-f-sharp-001.mp3
marimba-variation-v2-f-sharp-002.mp3
marimba-variation-v2-f-sharp.mp3
marimba-variation-v2-g-001.mp3
marimba-variation-v2-g-002.mp3
marimba-variation-v2-g.mp3

Ashton-Morris commented 4 years ago

My own personal notes have this "Create two versions of the C G C G organ sound and send that in" I think I was going to create a second success variation so it wouldn't sound too repetitive.

@zepumph The dev build wasn't published, can you let me know which organ sound we had in use so I can create a variation of that?

zepumph commented 4 years ago

From design meeting today:

[x] We want staccato sounds always playing when you are holding a hand with the mouse. For keyboard, we think that perhaps you have to not just focus, but then move once for this behavior to then be constant until next blur.
[x] We decided to go with the staccato sound for out of proportion feedback, and to use the "random mix" as the sound for it. All else can be cleaned up!!!
[x] Add hysteresis to the success sound, so that it doesn't play too often when going in and out to of Proportion quickly.
[x] @Ashton-Morris to create two versions of a success sound modeled off of success-indicator-elec-piano-2w.mp3 from https://github.com/phetsims/ratio-and-proportion/issues/9#issuecomment-651785405. We want the sound slightly shorter in attack (even though it will still ring out), and we want at least two versions so that we don't have the repetition of the same sound over and over (the JT principle).
[x] Change the default to the "ding" success sound so that we can see how we feel about that this week. This is in spite of the fact that we likely will choose an arpeggiated sound due to its ability to "stand out" better.
[x] (see https://github.com/phetsims/scenery-phet/issues/612) Add "slider bump" sound, also used as the ratio hand passing a grid node. Make this the sound for NumberSpinner. We will start by adding it to the common code. I'll create a separate issue for that.
[x] @Ashton-Morris will prototype some AccordionBox sounds, as we don't like that it shares the combo box sound currently.
[x] ComboBox sound: @jbphet will use the sound generator from Radio buttons for the default selection sounds (so that it differs for each item).
[x] @jbphet to: In RaP, override the challenge select ComboBox sound with the "best of" from molarity to make it shine and pop and stuff.
[x] @jbphet to add the "generic" button sound to the phet button.

zepumph commented 4 years ago

To discuss for next meeting from me:

I added hysteresis for the in proportion succes sound, and it can be adjusted with ?hysteresisThreshold. The default value is .1, which means 10% of the total fitness range above and below the in proportion range.
I had a question about the random mix staccato sounds. Would it be ok if they could repeat? There is code that makes sure of no repetition, but I wasn't sure if that was necessary.
Question in https://github.com/phetsims/scenery-phet/issues/612#issuecomment-657710112

I may add more in later.

zepumph commented 4 years ago

I have implemented all my stuff for this issue, and will continue to work on NumberPicker sounds over in https://github.com/phetsims/scenery-phet/issues/612. Unassigning until Tuesday.

Ashton-Morris commented 4 years ago

Here are some faster electric piano sounds with a faster attack

sucess-indicator-elec-piano-2w-faster-attack.wav
sucess-indicator-elec-piano-2w-faster-attack-001.mp3
sucess-indicator-elec-piano-2w-faster-attack-001.wav
sucess-indicator-elec-piano-2w-faster-attack.mp3

Ashton-Morris commented 4 years ago

I tried to create a woop like John suggested.

accordian-box-close.mp3
accordian-box-close.wav
accordian-box-open.mp3
accordian-box-open.wav

I also tried sounds that were closer to a real Accordion but I didn't bounce them because I feel they were too extreme sounding for PhET.

This approach was not appreciated the last time we were creating sounds for opening and closing but I added some white noise sweeps just in case. This is a common UI sounds used in applications and I thought it was a good reference point

accordian-box-close-white-noise2.mp3
accordian-box-close-white-noise2.wav
accordian-box-open-white-noise.mp3
accordian-box-open-white-noise.wav

All in all I think we'll need to iterate on this to get closer to the correct sound.

jbphet commented 4 years ago

Issue for adding sound to PhET button: https://github.com/phetsims/joist/issues/637

zepumph commented 4 years ago

From design meeting today:

"In Proportion" sound:

New sound option that only uses roots and fifths (perhaps starting at g4 since that is where the staccato high note is).
new option that is a major chord (perhaps starting a third above c4 to not start on the same note as the staccato sound).
Make at least three options.
Try making this higher than the sounds from this week (in https://github.com/phetsims/ratio-and-proportion/issues/9#issuecomment-658288530). We do not want it as high as the glock overtones from glock-marimba-c-maj-7-arp
Still make these ring out, we will fade them out with WebAudio when moving out of Proportion.

"Moving in Proportion" sound:

New options should be like choir but more synthetic (less human but not "jump" by van halen)
Create at least 2 new options to explore this.
Start at g4 (above the starting in Proportion sound)

We want these options created and delivered by Monday so that I will have time to input them before the next meeting which will be either next Tuesday or Wednesday.

zepumph commented 4 years ago

I have nothing to do in this issue until there are sounds for me to add to the sim. Unassigning.

Ashton-Morris commented 4 years ago

"We want these options created and delivered by Monday so that I will have time to input them before the next meeting which will be either next Tuesday or Wednesday." Just read this again. Hope you get time! "In Proportion" sound: in-proportion-fifths-option-1.wav in-proportion-major-chord-option-3.mp3 in-proportion-major-chord-option-3.wav in-proportion-major-chord-option-2.mp3 in-proportion-major-chord-option-2.wav in-proportion-major-chord-option-1.mp3 in-proportion-major-chord-option-1.wav in-proportion-fifths-option-3.mp3 in-proportion-fifths-option-3.wav in-proportion-fifths-option-2.mp3 in-proportion-fifths-option-2.wav in-proportion-fifths-option-1.mp3

"Moving in Proportion" sound: moving-in-proportion-loop-option-4.wav moving-in-proportion-loop-option-3 .wav moving-in-proportion-loop-option-2.wav moving-in-proportion-loop-option-1.wav

zepumph commented 4 years ago

All options above have been implemented. They can be found in this dev link. Looking forward to discussing them tomorrow!

https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.38/phet/ratio-and-proportion_en_phet.html

zepumph commented 4 years ago

Discussion about the Moving in Proportion options:

[x] Make the moving in Proportion sound mutually exclusive with the staccato sounds, implementation wise this means making movingInProportionThreshold and MOVING_IN_PROPORTION_FITNESS_THRESHOLD add up to 1. Already implemented in https://github.com/phetsims/ratio-and-proportion/commit/77f0c610f1c41928c87f166460a1ec64da24fd73, but should be cleaned up to just use a single value.
[x] Create a sound option that includes both the "peaceful tranquility" and the "choir sound"
[x] Only keep four options, the aforementioned mix, choir, and "peaceful tranquility," and "dream time"
[x] @Ashton-Morris to create a sound that layers the choir sound plus a rendition of "peaceful tranquility" (moving-in-proportion-loop-option-4.wav), and removes a few of the lower bass notes so that it floats above the in proportion sound. (less breadth to the sound).

Discussion about the in Proportion options:

[x] The hysteresis option is weird when just moving a single value, as fine tuning it can yield just a flash of green, and no sound. Perhaps it should only be implemented for when moving in proportion.
[x] Get rid of both "option 3"

Thoughts about how the moving in proportion sound relates to the in proportion

Maybe the in proportion sound can be swapped out for the moving in proportion sound, instead of layering on top of.
Perhaps we can just make the in proportion sound quieter when the moving in proportion sound is playing
Maybe we can make the moving in proportion sound actually include themes or elements of the in proportion sound in it (and to just play that). Note the complexity of adding in a single ringing "ding" into a looping sound.

These are all thoughts, with no direction just yet on what to implement. @BLFiedler and @emily-phet will take a look at the sim tomorrow and see if there are any implementation options to try to add. Keeping in mind that this is in addition to the change I just made in https://github.com/phetsims/ratio-and-proportion/commit/77f0c610f1c41928c87f166460a1ec64da24fd73

The goal for next week's meeting is for everyone to come to it with a combination of sounds that they think works well together, and for us to discuss which ones to implement as sets, instead of just individual sounds.

emily-phet commented 4 years ago

@zepumph @BLFiedler - I played with the sim this morning quite a bit. I think the dev 40 paradigm for mutually exclusive in proportion, success, and choir sounds is good.

I think the choir sound could tail off faster. Sometimes it feels like I can still hear it even if I'm quite far off the ratio position.

If @BLFiedler agrees, I think we move to all using the sim with the different sound combinations, and listing out preferred options (considering all the sounds together).

brettfiedler commented 4 years ago

Yeah - I think this is plenty smooth. The staccato and in-prop sounds seem to play when you move far enough out and/or re-enter just fine.

re: choir sound tailing off: I think this is particularly noticeable when you lock the ratio. There is a time delay between ending movement and the choir sound stopping. I agree that delay could be shorter so it's not still playing if you stop or leave the proper proportion.

Side note: @zepumph, I discovered if you select the Peaceful Tranquility velocity sound and then hit the reset button (iPad, Safari) it locks all interaction with the simulation. Only that sound option though. Made an issue: #97

zepumph commented 4 years ago

Side note: @zepumph, I discovered if you select the Peaceful Tranquility velocity sound and then hit the reset button (iPad, Safari) it locks all interaction with the simulation. Only that sound option though. Made an issue: #97

Should now be fixed

I think the choir sound could tail off faster. Sometimes it feels like I can still hear it even if I'm quite far off the ratio position.

This could be a bug, or could be our current fade out time of .2 seconds. I'll try decreasing that to .1 to see if that is better. I'll get back to you after some investigation.

All other sound options have been implemented on master.

emily-phet commented 4 years ago

Sounds like we're all set to come up with our list of favorite combinations, in preparation for Tuesday. I'll let all know to do this in the RaP slack channel. :)

emily-phet commented 4 years ago

Note - there are still some remaining to dos for @zepumph and @Ashton-Morris in this: https://github.com/phetsims/ratio-and-proportion/issues/9#issuecomment-662725606 above.

Ashton-Morris commented 4 years ago

I have this here. ^__^

moving-in-porportion-loop-preaceful-and-choir.wav
moving-in-porportion-loop-preaceful-and-choir.mp3

emily-phet commented 4 years ago

My thoughts re staccato success sound and velocity sound.

I'd be fine with any combination of "Fifths option 2", "Chord option 2" for the staccato success sound, and "Choir" or "Peaceful + Choir = nirvana" :) sound.

My top two favorites are: 1) Fifths option 2 + choir 2) Chord option 2 + choir

Some thoughts on these preferences:

I like the fifths option 2 slightly more than chord option 2 for this purpose because fifths sounds a little more "smooth" and chord rings out really bright and loud (in a good way, but it does stand out a lot).
I like choir slightly more than choir+ because it's a bit simpler, and doesn't feel as "full" as the choir+ sound, which (to me) may sound a little odd if you're trying to use the feedback in a group setting turned down low. Since it's so "full" sounding, it feels to me like it's asking to be listened to louder and as more of the focus for some reason.

emily-phet commented 4 years ago

@zepumph There were times when I thought I was hearing sound feedback when the screen looked white. Is this the case, or is the tolerance such that the screen just happens to look white because it's a super subtle shade of green? If it's the case that the background is exactly white but providing sound feedback, we should discuss if we want this to happen tomorrow.

zepumph commented 4 years ago

After https://github.com/phetsims/ratio-and-proportion/commit/3523bc99cd94018890f94f2c585e5d4a2a4526f9, now the ?movingInProportionThreshold query parameter takes the value in "distance from max fitness" instead of distance from min fitness. So the value used to be .7, but now it is .3.

zepumph commented 4 years ago

Notes from adding the new combined wav sound:

I added the new wav file from @Ashton-Morris and named it moving-in-proportion-loop-peaceful-and-choir.wav.
I can't actually hear a difference between the two sounds I combined in code, and @Ashton-Morris's. Did you take out some of the bass notes as described in https://github.com/phetsims/ratio-and-proportion/issues/9#issuecomment-662725606? I think my ears may be failing me.

zepumph commented 4 years ago

Everything has been implemented from the above checkboxes.

If it's the case that the background is exactly white but providing sound feedback, we should discuss if we want this to happen tomorrow.

I have not seen this, but will be on the lookout. Please create a new issue if you can consistently reproduce. I'm pretty sure that it is just a very faint shade of green. Hopefully logging added in #103 will help see that.

My thoughts about combos:

I agree with @emily-phet that option 2 is a bit better than option 1, and I like fifths more than chords (yay fifths 2)!
I like the choir+ sound most.
Thus my vote is fifths2 + Choir+ peace
I think that fifths 2 could use some minor tweaking to have a bit less twang at the end. It is interesting on its own, but it is a bit muddy when combined with the "moving in proportion" sound. I feel this may no matter what moving in proportion sound is selected.

Looking forward to tomorrows chat!

jbphet commented 4 years ago

As requested by @emily-phet, I spent some time reviewing the latest dev version (https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.41/phet/ratio-and-proportion_en_phet.html) and have the following comments:

The hysteresis for playing the "in proportion" sound is too much when not doing dual drag. I know I advocated for having some hysteresis, so I should elaborate. When using the sim on a touchscreen and moving both markers, the "in proportion" sound played and stopped a lot, and this seemed distracting, so hysteresis seemed important. However, when using v1.0.0-dev.41 with a mouse, it feels odd that the sound isn't played each time the optimal proportion is reached and the max green is shown. So, perhaps the hysteresis should only be used when both markers are being dragged, or maybe the amount should be much less when one marker is dragging versus both markers. This problem was particularly noticeable when using fine-grained keyboard-based movement (shift-arrow).
The "in proportion" sound should play every time the threshold is crossed. As it is now, some fast crossings don't cause it to play.
My favorite combination of sounds is also "Choir/peaceful tranquility blend" and "Fifths option2". The former seems like the clear winner to me for the "Velocity Sound". For the latter (currently labeled "Staccato Success Sound"), I would also be okay with "Arpeggiated maj7 chord" or "Single ding high note".

zepumph commented 4 years ago

The hysteresis for playing the "in proportion" sound is too much when not doing dual drag. and The "in proportion" sound should play every time the threshold is crossed. As it is now, some fast crossings don't cause it to play.

This was fixed in master in https://github.com/phetsims/ratio-and-proportion/commit/b643b7a894efc85a2c3900d42cdbaa8516e9c034 and was a bullet from a previous design meeting, see https://github.com/phetsims/ratio-and-proportion/issues/9#issuecomment-662725606:

The hysteresis option is weird when just moving a single value, as fine tuning it can yield just a flash of green, and no sound. Perhaps it should only be implemented for when moving in proportion.

brettfiedler commented 4 years ago

The velocity sounds are reaaalllyyy quiet for me relative to the in/out of prop sounds, so this was a little harder to assess how rich/crowded the soundspace was, but:

MovingInProp: Choir/Peaceful Choir are on nearly equal footing for me. I can see how the soundspace could be a little overpacked. I'd be fine with either. I suspect those using external speakers won't really hear the added parts in PeacefulChoir.
InProp: I am fine with either Option 2 or the Arp Maj 7th. I might like the Arp Maj 7th the best if the notes the choir was singing meshed up a bit better? Something about the softness of that success sound is still appealing to me for repeated listens.

Could we maybe consider playing a minimum amount of a success sound that we consider the "core" part before it rings out? It does feel a little weird to let go and have some of the arpeggiated sound cut out so quickly. This may also address John's comment on sounds sometimes not playing. I suspect it may be triggering, but getting cut off just as quickly?

zepumph commented 4 years ago

From sound design meeting today:

[x] Moving in proportion sound is much quieter than the in proportion sound. We should balance those by probably meeting somewhere in the middle.
[x] We decided as a group on Choir + Peaceful and Fifths 2. It may need some fine tuning, but we want to go through a round of interviews before doing more work on this (in case we need to pivot).
[x] Clean up all other sound options
[x] (not needed for interviews) When you would hear the staccato sound, don't play pick up and drop UI sounds for the hands.

zepumph commented 4 years ago

Some notes to look at when (if) we come back to fine tune the Choir + Peaceful and Fifths 2 sound combo:

For now we are keeping the bassy breadth of the peacful tranquility sound in because it sounds "complete," but in the future, we may change how we feel to thinking it a bit too muddy. We will come back to this thought
The balance of levels between Moving in proportion sound vs in proportion will likely need to be tweaked beyond what is done for https://github.com/phetsims/ratio-and-proportion/issues/9#issuecomment-665287475.
We may want to make tweaks to the fifth option 2 sound to blend better with the nirvana sound.

zepumph commented 4 years ago

Once the above is implemented, we can close this issue. As stuff comes out of interviews, we can make more issues for it.

zepumph commented 4 years ago

Alright! All has been implemented and cleaned up. Let's create new issues from here as we fine tune our sounds. Good work team!

phetsims / ratio-and-proportion

Implement sound design #9