Implement sound design

[jbphet]

@Ashton-Morris, @emily-phet, and I (@jbphet) did some brainstorming on the sound design for the proportion sim and thought we should write some of these things down.

• Use two sound generators that are producing the same note when in correct proportion, and when the proportions aren't correct they would produce different notes and thus beat frequencies.
• Same idea - two sound generators - but produce the notes periodically so that it doesn't drive people crazy with continuous sounds
• Two rhythmic beats that are distinct and line up when the proportions are correct - in a fun way, using double time, triplets, etc, but get out of sync if the proportions are incorrect. They'd speed up or slow down the overall tempo as the fundamental sizes change.

[jbphet]

I made a quick recording to demonstrate beat frequencies. In the explanation at the beginning I probably should have said that I'm "playing the same note on two different strings" instead of that I'm "playing the same note".

Here's a link: https://drive.google.com/file/d/1AmrwecgS4_QDzztTGWdeK7pM2TMpIomE/view

[zepumph]

In the above commit. I made a refactor that exposed what I call the proportionFItnessProperty in ProportionModel. This is a value between 0 and 1. When 1 the proportion is "perfect", and 0 means it is outside of the tolerance range. I did this with the hope that it would be easy to monitor for a simple "fitness" sound that can change in the same way as we do for color. Let me know if I can be of more help.

In my opinion having something (even very simple) implemented by next meeting seems very valuable for discussion. Good luck!

[emily-phet]

@Ashton-Morris - please propose something (could be related to beat frequencies or anything else) that is reasonably pleasant and can be added to the sim quickly. I'd like to get some sound in asap so there's some auditory feedback available as the team brainstorms sound and other features. Let's start with something that aligns with the color change in the sim, as @zepumph indicated.

@jbphet @zepumph Please work together to add the first sound. @jbphet if you're able to begin showing @zepumph the ropes on how to add sound to sims with Tambo, that would be ideal, but if depending on where you're at with your other rocks, you might do this later in the quarter and proceed with whatever is fastest now.

[Ashton-Morris]

@emily-phet I have a few ideas ready but I cant seem to properly clone the directory that has Proportion.

It would be much quicker if someone could do a simple screen recording of a proportion slider slowly going up and down. I can then do a few mockups of how sound would work and can upload it here.

If anyone is willing to do that please let me know.

[emily-phet]

@Ashton-Morris - I'll try to do a screen recording for you later today. Here's a link to the latest dev version, in case that's helpful in the meantime. https://bayes.colorado.edu/dev/html/proportion/1.0.0-dev.5/phet/proportion_en_phet.html

[Ashton-Morris]

@emily-phet Don't worry about a screen recording, I can do that. I didn't know there was a dev version. That all I needed.

I'll work on some mockups asap

[emily-phet]

@Ashton-Morris I figured that might have been the issue. :)

When you make the screen recording, keep in mind that in some scenarios, learners will be moving each object one at a time, but in the more desirable scenarios, they're moving both at the same time (e.g., on a touch screen, or with tangible objects). Just thought that might impact your thoughts re sound.

[Ashton-Morris]

I worked on some ideas this morning. You should be able to view these in your browser. I will do my best to explain my thinking with each one. @emily-phet Please assign the proper people for review

• Proportion_C_Major_Sine.m4v Max out of proportion = Low volume and only a C sine wave tone Max in proportion = Added E and G notes that fade in which creates a C Major Chord What you're hearing = A C Major Chord fading in and out along with proportion

• Proportion_Fifth_Sine.m4v Max out of proportion = Two C Sine Waves and No Volume Max in proportion = Two Sine waves C and G creating a perfect fifth What you're hearing = A kind of wobble/dissonance as they are out of proportion and a cleaning up/harmony sound when they are in.

• Proportion_One_Semitone_Sine.m4v Similar to above Max out of proportion = Two Sine Waves one C and one B and No Volume Max in proportion = Two Sine waves playing the same note C What you're hearing = A kind of wobble/dissonance as they are out of proportion and a cleaning up/unity of the notes sound when they are in.

• Proportion-Rhythm.m4v Max out of proportion = A single rhythm at a low volume Max in proportion = That same rhythm being played at 2X and 4X the speed fading in in volume What you're hearing = A kick drum like sound going by it self then a more complex rhythm rising in volume as Proportion is reached.

• Proportion-Vibrato.m4v Max out of proportion = A single tone with vibrato and a low volume Max in proportion = A single tone with no vibrato What you're hearing = A single tone going from wobbly/vibrato to straight and pure and louder as Proportion is reached.

• Proportion-Random-Clicks.m4v Max out of proportion = A single repeating thump/click that has it's pitch randomly changed by 16 semitones each time it repeats at a low volume Max in proportion = A single repeating thump/click playing a consistent rhythm at a louder volume What you're hearing = A single repeating thump/click that goes from being in disarray (playing random versions of itself) then fading into a consistent rhythm when the sim is in Proportion.

[emily-phet]

@Ashton-Morris Thanks for these! @zepumph @jbphet Can we have a version with two sound options - with "Proportion-Vibrato" and "Proportion-Random Clicks" (random clicks as the default)? If it's not time consuming to do, having a third option of "Proportion_C_Major_Sine" would be nice to have in as well.

To ensure those playing with at-home tangible versions has some auditory feedback to consider, this would be my top priority effort for Proportion development this week.

[emily-phet]

@Ashton-Morris Here are my thoughts:

• C_Minor_Sine - this sounds very pleasant to my ear. My concern is that from the video I wasn't sure if the feedback tied strongly enough to the goal state of "at ratio". That state needs to simultaneously stand out clearly. If this one gets implemented in the first options set, my hunch is that it will feel like it stands out more so than watching the video. Let's see!

• Fifth Sine and One_Semitone_Sine - both of these sounded slightly less pleasant to my ear than C_Minor_Sine, and had the same potential issue of not quite strong enough of a highlight of the "at ratio" state.

• Rhythm - I thought this approach might muddy the waters a bit at cueing the 'at ratio' state. If we feel like pursuing a rhythm-based approach makes sense to move forward with, I think we'll want one with a really strong change when very very close, and right at, ratio.

• Vibrato - I thought this one had the strongest cueing to "tune" the ratio, though slightly less pleasant to my ear than the C_Minor_Sine, and the random clicks.

• Random Clicks - I really liked this one, and I'm curious what others think. It sounds interesting and pleasant, while also providing some strong cueing when you're "at ratio".

[Ashton-Morris]

I have a single click and a loop of them below for implementation.

In my DAW I have the clicks set to randomize between 0 and +16 semitones. The Randomizer's rate (like the rate of an LFO) is set to 30HZ. Then its set to 100% out of proportion = 100% of that randomizer effecting the clicks, and In proportion = 0% that randomizer effecting those clicks.

proportion-random-clicks-loop.mp3 proportion-random-clicks-single.mp3

For the Proportion_C_Major_Sine you could play sine's with web audio. That would be: Play C3 when it is 100% our of proportion and then attach the volume of the sine waves E3 and G3 to how in proportion the sim is. Thats how I had it set up.

Edit: These things can be tricky with web audio, we can talk about implementation hurdles later today ^__^

[zepumph]

I, with tons of great help from @jbphet, added modes for the vibrato and random clicks sounds. The random clicks is on by default. The vibrato is currently just two sine waves at 220hz where one can be moved up to 230hz depending proportion fitness.

[zepumph]

After the above commits, we successfully have mechamarkers input working with the two sounds discussed during today's meeting.

Summary of what is currently in the sim:

• A prototype of Vibrato
• A prototype of Random Clicks
• A prototype that just changes pitch based on the fitness
• Sound working for keyboard/drag/marker inputs

Next I will try to add C_Major sine as the final step for the day.

[zepumph]

Above I added the CMajor sine wave generator. It sounds really fun. I'm excited to be working on the sound design for this sim. I got perhaps a bit carried away and added a high C (C4) only when the proportion was perfect. This made for a direct cue from sound for when the user got it totally right. I'm not sure that this implementation is the best, but I really like a specific sound for when (and only when) the "fitness" of the proportion is exactly perfect (in the model a value of 1).

Dev version coming soon.

[brettfiedler]

@terracoda noted that the sound plays continuously on keyboard focus of either cursor without directional input.

It's not clear what the preferred feedback should be right now, but something we should decide at some point.

[zepumph]

Yes that was done purposefully. After I tried it both ways (based on focus/blue and based on keydown/keyup), it felt the committed way was much easier to tell if there was success or not because it was much more continuous.

[brettfiedler]

A note from an interview using 1.0.0-dev.9:

• With Random Clicks enabled, the user thought they perceived an increase in the speed of the clicks even when they had not met the threshold for the increase in frequency to begin. Looking for signal in the noise type situation.

Update: Note from an interview using 1.0.0-dev.10:

• Noted the increase in regularity/frequency of clicks and used it to locate green success regions
• Found the sound options on her own: Liked the single pitch the most. Thought random clicks get a bit annoying after a while. Related single pitch to a metal detector.

The sound design strategy is a success for helping them locate the proportion though! The presence of the sound has been remarked as helpful by all interviewees (x3)!

Other general questions I'm not certain about, but come to mind from watching learners:

• Can the sound help learners maintain the proportion?
• Should different proportion selections have different qualities to the sound?

[emily-phet]

@BLFiedler Glad to hear the sound cueing seems helpful for finding the target ratio! Please create a spreadsheet where you can summarize your findings for each interview, regarding specific categories - visual display, auditory display, interaction pathway, etc. You'll also want some fields for sim version, device, and some basic demographics. Let's add on to this as we do more interviews. You might reach out to Brianna (she's on Slack) for an example of some spreadsheets she'd made in this ballpark.

Also, can you 1) clip out the proportion section of these interviews, and 2) edit out all screen area that is not the sim (e.g., people's browser tabs, etc.). Sometimes there is identifying info in the top of the browser window. You can use Camtasia to do this really fast. Let me know if you need some help with this. 3) Put all of these de-identified clips in a shared folder, for the team.

[emily-phet]

@jbphet @Ashton-Morris Just a heads up, we'll be collecting more feedback re sound from the design team this week. So for this week sound is otherwise on hold, unless there are some specific topics you'd like to bring up when we meet on Tuesday.

[emily-phet]

A few thoughts on the random click sound - after playing with the sim + mechamarkers for a bit, I think the random click sound is the least helpful, and most challenging to listen to after awhile. I think tuning more when the change in rate happens will help. Thought not sure if it will help enough to put this sound as the frontrunner. Implementation of the grid representation could help with the "tuning" process for this sound, which may happen this week.

[emily-phet]

More thoughts - we should probably start a sound design doc to put these.

• I have liked the vibrato sound the most while playing with Mechamarkers. The current sound is good at indicating nearish-vs-not, but once you hone in on "keeping the screen green" while moving the markers up and down, the sounds don't help much. We need the sound to indicate 1) clearly when exactly at ratio, and to 2) change significantly in the range of close-very close -super close.

[emily-phet]

@zepumph Can you make "Vibrato" be the default sound (leaving all current options)?

[zepumph]

Done above.

[Ashton-Morris]

I came up with more examples of how it could sound when you are in proportion but you are also staying in proportion and moving up and down. @emily-phet please assign people as necessary.

• Proportion_Applause.mp4
• Proportion_FM_Synthesis.mp4
• Proportion_Major_Chord.mp4
• Proportion_Moving_Sine_Example.mp4
• Proportion_Stacc_Sine.mp4
• Proportion_Stacc_Sine_w_reverb.mp4
• Proportion_Strings.mp4

[emily-phet]

@Ashton-Morris These are wonderful! I love these staged approaches you're taking.

I believe we want learners to experience this sequence: 1) something going "out of phase" then "into phase" [corresponding with the sim starting at ratio (good), then going out of ratio as the learner first moves the object, then going back to ratio (good) as learner nears same or new ratio arrangement] They'll hear this multiple times, as the move the markers around exploring the space. 2) going from "in phase" (good) to "better" [corresponding to being at ratio, then moving the markers such that markers stay at ratio]. In your demos above, all had a feeling of transition in them, but not all had a clear feeling of going from "good" to "better". Most had a feel (to me) of going from "good" to "different".

Proportion_Strings and Proportion_Applause had the most "good" to "better" feel to me. The transition to the strings in Proportion_Strings felt like a bonus, as it's recognizably going from synthesized techo-sound to a recognizable instrument (which can be easily talked about verbally and is also easier on the ears).

The transition including the applause in Proportion_Applause was also clearly cueing "good" to "better", once it was clear you were hearing applause. When it first starts (and for learners it would likely start/stop/start/stop for a bit until they get the hang of the movement), it's not 100% clear it's applause. But the basic idea of it was very nice.

It would be great to hear the following:

• an updated demo of Proportion_Strings with applause coming in about halfway into the strings. (I'd like to hear if that feels like overkill altogether).
• 1 or 2 more demos like Proportion_Strings, but with other timbres (instruments) instead of strings. Think things that middle schoolers might feel good about getting as feedback. If in a classroom, we'd want them to be saying, "hey, how do I get it to make that sound" when they hear a neighbor getting the "better" sound.
• A demo similar to Proportion_Stacc_Sine that starts with staccato, then becomes strings.
• A demo similar to Proportion_Stacc_Sine that starts with staccato, then becomes strings, then layers in applause over the strings.

And, because I know you like a challenge - double bonus if any of these can include a cello sound. Dor (father of this sim idea) plays the cello. :)

This Tuesday, let's discuss how this kind of staged sound could be implemented in the sim. I'll ask @zepumph to join for part of our Tuesday sound design meetings.

[emily-phet]

@BLFiedler please add any thoughts you have as well.

[brettfiedler]

Stage 2 (in prop): My general impression is that the movement toward a neutral, single tone (from something else) is very successful in generating that feeling of "I found it!". This sound can possibly be more pleasant to listen to for a long period of time, but I don't think it HAS to be. What we need it to be is pleasant enough, but not as nice and rewarding as the next tone. This leads me to...

Stage 3 (maintain prop): I think Emily mentioned this, but overall this sound should be BETTER (oh my, subjective), than the in proportion sound. This may be richer or more pleasant or runner, but also should not at all sound like the sounds we heard before (out of proportion or searching sounds). Some of the examples re-introduce some reverb/vibrato (I apologize for confounding those two), which to me may cue the learner to believe they've left the Success they have already achieved, when in fact they've reached an even higher success. I like the Strings for that. I like the idea of the applause for the same reason, but personally find it a little jarring.

I'm going to distinguish between two phases in the Out of Proportion (Stage 1)

Stage 1A (way out of prop): I think both the kind of reverb-y vibrato, as well as the staccato frequency, are successful in generating that sense of "I'm searching for something and it's guiding me toward it". Personally, I find the staccato effect to give stronger feedback for distance from proportion and the sense that I am looking for something.

Stage 1B (nearing prop):

1. The dissonant sine waves "slow down" and resolve into the success tone, which I think is smooth and "feels good". However, it's a little muddy when you're near the proportion but not quite on it.
2. The staccato frequency becomes very rapid and the success tone fades in. Without playing sim version with this implemented, I get the sense that this is a better contrast from out of proportion to in proportion. I would actually like to hear a version where there is a more rapid fade-in of the success tone (though we may need to consider tolerance and the visual fade-in as well?)

[Ashton-Morris]

@zepumph I have some files for you which are all stored here. Some of these are .wav so we can have better loops during the prototype phase.

I added a staccato sine but realistically that's something we'd need to do in web audio programmatically.

applause-loop.wav staccato-sine.mp3 strings-loop-c5.wav cello-loop-c4.wav synth-strings-loop-c4.wav bowed-piano-loop-c4.wav cello-tremolo-loop-c4.wav

[jbphet]

I listened to the video demos after today's sound design meeting and agree with what was essentially decided in the meeting, which is that we should prototype the "Proportion_Strings" version and try adjusting some of the tolerance parameters to see if we can get it to work in a way that is viable.

[jbphet]

I'm going to unassign myself for now, since I don't think there is anything specific that I need to contribute at this point. @zepumph - I'm happy to help with the implementation if I'm needed, just assign it back to me if so and/or we can set up a time to collaborate on it.

[zepumph]

After lots of design discussion today, with @Ashton-Morris playing many possibilities for us, we have decided on the following for the next steps:

• [ ] Add the chorus option for the velocity sound, (currently strings)
• [x] Add a new ratio sound of marimba tones, increasing in frequency (how often it is played) the closer to successful ratio it gets. Then at a threshold it turns into a root/third tremolo loop. I will take the marimba sound from Tambo. (use bright-marimba.mp3, the "short" is available if they are bleeding into each other).
• [ ] We will also add an option, same algorithm as above, but with pizz strings instead of marimba.
• [x] We want to "play with silence," and to remove the continuous sound when the ratio "fitness" is equal to 0. This means we only get sound when you are "getting close" to the ratio. Tagging @emily-phet to see if she disagrees.
• [x] We will hold off on fading the ratio sound out to bring the choir sound in. This fine tuning can be done later, once sounds are more solidified.
• [ ] @Ashton-Morris will send the Choir sound (to be looped) as an mp3 and wav. I will try using the mp3 first to see if it works, otherwise we will use the wav and take the size hit.

[zepumph]

Marimba staccato plus tremolo to third is in place on master and the default sound. I also have turned off sound when the fitness is at 0. Next steps are to add in choir and pizz sound options once @Ashton-Morris has them ready. Over to him.

[emily-phet]

@zepumph re

We want to "play with silence," and to remove the continuous sound when the ratio "fitness" is equal to 0. This means we only get sound when you are "getting close" to the ratio. Tagging @emily-phet to see if she disagrees.

The plan sounds great to me!

[Ashton-Morris]

@zepumph I have the files for you here

choir-ahhh-loop.wav pizz-C4.mp3 pizz-C3.mp3

[emily-phet]

To discuss on Tuesday:

• Adjusting rate of staccato (current version sounds a bit frantic) @zepumph can you make sure it's straightforward to change/tune the tempo of the marimba/pizz sound?
• Consider reversing the mapping, where tempo decreases markers approach success. This require sounds to be playing whenever the markers are in use (which we decided last week we didn't want) but I think this is useful to consider even if just conceptually, especially if we're become less confident that the current mapping (tempo increases as markers approach success) will be effective without the sound seeming frantic as success is reached.
• Consider what the sound is when you reach ratio - what will feel satisfying, perhaps calm, and confirm success. Could this be a single sound, different pitch than sounds heard on approach, and then (if not moving markers continuously) silence?
• General question - what is the smallest tempo change that is readily recognizable?
• For the sound played during continuous movement with success (currently, the choir sound), consider which gives effective cueing, while being the most pleasant - Continuous, layered, or looping sound for sound.
• If there's time in the meeting, can also consider sounds to remove from current sims sound options list (perhaps the first and second options, at least?)

[zepumph]

I added pizz staccato options and the choir ahh velocity option. See master or https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.21/phet/ratio-and-proportion_en_phet.html. Sorry to cut it short to our meeting time!

[zepumph]

Nodes from today's design meeting

• Behavior of when the velocity sound plays. Right now it can occur when any staccato sound plays (any fitness >0), even when the tremolo to the third isn't going. @jbphet mentioned it may be nicer to layer on the velocity sound only when you maintain the tremolo sound while moving. If that is the case we need to greatly relax the threshold for the "tremolo to the third" sound. Right now it has to be pretty much a "perfect" ratio.

• Can we reduce the frequency of the tremolo (right now it is every 40ms).

• @emily-phet asked what can we do to simplify these cues. Less "noise". Increasing frequency until at the right ratio, then a "release" with less sound

• Explore a single tone at the end, perhaps with some effects/envelope to make it longer (but not continuous). This keeps playing, but is shaped differently than the unsuccessful sounds.

• Single, new, "success" sound. Like a gong. Only once until to leave success and come back.

  • Could be a higher "ding" like a cash register to indicate success.
  • A long, reverby gong (we won't do this).
  • A chord or lightly arpeggiated group a notes.

Action steps:

• [x] we can get rid of the following sound options:
  • Random clicks
  • Vibrato and strings
  • Single Pitch change
• [x] add a "No Velocity sound" option
• [x] @Ashton-Morris to get new pizz sounds with slightly less reverb.
• [x] New sound options for staccato success:
  • tremolo thirds (current)
  • single note (higher and longer tail) no tritones allowed
  • with chords, try (1,3,5,7Maj) and maybe we go with (1,3,5,8)
    • single chord
    • single arpeggiated chord

[zepumph]

Clean up above.

[zepumph]

Above I added staccatoMaxFrequency, so that we can prototype the tremolo speed. I also doubled it from 40ms to 80. Here is an example of it doubled even again:

http://phettest.colorado.edu/ratio-and-proportion/ratio-and-proportion_en.html?ea&brand=phet&staccatoMaxFrequency=160

[zepumph]

Above I added chords and arpeggio staccato success sound options. All of these have some build in reverb that @jbphet helped me get set up using soundManager.reverbLevel. I don't know that it will be a long term strategy, but it seems valuable to have to this week's testing.

I will make a dev version shortly to share this work.

[zepumph]

https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.22/phet/ratio-and-proportion_en_phet.html

[emily-phet]

Thoughts:

Staccato Success Sound:

• tremolo to third - a bit chaotic, doesn't lead me to think there's a single "special" location
• single high note - I liked this one best, but I think the single sound needs to feel richer somehow, and stand out more.
• single major 7th chord - same as below
• single major chord - current variant feels like a little 'bump in the road', doesn't stand out enough
• single arpeggiated chord - too long, I think

Ratio Fitness Sounds: The pizz sounds sound better to meet at the high rate as you approach success, but I like the individual marimba sounds a bit more. Let's try bringing the max rate for all staccato "Ratio Fitness Sounds" sounds down by 50%, and expand by ~25% the range at which you can hear these sounds. @zepumph let me know if you have any questions on this.

[zepumph]

single arpeggiated chord - too long, I think

I decreased the time of arpeggiation from .25 seconds to .1 second, and I like it a bit more. Actually I think it is my favorite staccato success. I feel like it helps solve the issue you had with the single "chord" strategies too. what do you think?

single high note - I liked this one best, but I think the single sound needs to feel richer somehow, and stand out more.

Agreed! I wonder if we could get a few options from @Ashton-Morris on "fuller" sounds to test out.

The pizz sounds sound better to meet at the high rate as you approach success, but I like the individual marimba sounds a bit more.

I think that perhaps we would like the marimba a bit more with less attack, and I think we would like the pizz a bit more with less reverb. Not sure which one I prefer.

Let's try bringing the max rate for all staccato "Ratio Fitness Sounds" sounds down by 50%

I doubled it from 80ms to 160ms. You can also fine tune this with ?staccatoMaxFrequency=X.

and expand by ~25% the range at which you can hear these sounds.

I increased the tolerance from .1 to .125, you can fine tune this with ?tolerance=X. Also please note that if this tweaking isn't cutting it, we can raise the priority and/or try to finish design on tolerance over in #14.

Please review.

[emily-phet]

@zepumph I tried, using the query parameters listed above, adjusting the tolerance (which worked for me) and the staccatoMaxFrequency (which I couldn't get to work). Can you confirm that this works, and also remind me of any symbols I need to include if using two query parameters (isn't there an ampersand needed or something?). Note, I was trying the query parameters individually just now, so combining two query parameters is not my main issue...

[Ashton-Morris]

Sounds Incoming

• glock-marimba-c-maj-7-chord-stab.mp3
• glock-marimba-c-maj-7-chord-stab.wav
• glock-marimba-c-maj-chord-stab.mp3
• glock-marimba-c-maj-chord-stab.wav
• glock-marimba-c-maj-arp.mp3
• glock-marimba-c-maj-arp.wav
• glock-marimba-c-maj-7-arp.mp3
• glock-marimba-c-maj-7-arp.wav
• c4-ding-ring-out.mp3
• c4-ding-ring-out.wav
• c4-marimba-long-ring-out.mp3
• c4-marimba-long-ring-out.wav
• marimba-c-maj-7-chord-stab.mp3
• marimba-c-maj-7-chord-stab.wav
• marimba-c-maj-chord-stab.mp3
• marimba-c-maj-chord-stab.wav
• marimba-c-maj-arp .mp3
• marimba-c-maj-arp .wav
• marimba-c-maj-7-arp.mp3
• marimba-c-maj-7-arp.wav
• pizz-C4-shorter.mp3
• pizz-C4-shorter.wav
• pizz-C3-shorter.mp3
• pizz-C3-shorter.wav

For the success sounds. I think the arpeggio works better than the chord stab because rising major arpeggios have been used to indicate success a lot in our culture/media. It's easier to instantly grasp

[zepumph]

@emily-phet I was able to reproduce your problem. I fixed it in master above. Please try the above link again, and note that the first query parameter gets prefixed with a ?, and each subsequent gets a &, so phettest can look like:

http://phettest.colorado.edu/ratio-and-proportion/ratio-and-proportion_en.html?brand=phet&ea&staccatoMaxFrequency=20&tolerance=.3

[zepumph]

There are a lot of sounds in. It would be good to get some feedback on them, as well as the ones in the sim already, before doing the next batch of iteration in the sim. I personally like how some of the sounds have some reverb at the end of the note/chord. I think that will work well if someone stays on the ratio, and then we can fade it out when going out of proportion. I think my favorite is the glock-marimba arpeggio. I think it could fit nicely with the choir velocity sound.

[zepumph]

That said, I will exchange out the pizz sounds, as we have already discussed that.

[emily-phet]

Here are some thoughts:

Staccato Max Frequency:

• For the marimba sound, I thought 50 is too fast, 100 is ok, 125 is ok (my personal preference when playing with sim), 150 is probably too slow. Let's go with 100 for now. Better to err on the slightly too fast side, I think, if we're between to favorites. With pizz, everything sounded a bit faster than marimba (like an auditory effect, probably not actually faster). We may want to tune the values slightly slower even, if we go with pizz or fizz-like sounds.

Timbre

• I prefer marimba over pizz, but wish it didn't sound so slot-machine like. Any chance @Ashton-Morris you have ideas for a relative of a marimba that has some of the same qualities, but is less slot machine-y when heard in repetition? For pizz - I like the pizz C3 better than pizz C4 now (@zepumph I think you called it last week :)).

For the sounds @Ashton-Morris added,

• I like the ding-ring-out for the success sound best (outside the sim). We know sounds can be tricky and not turn out how you think in the sim though...
• I liked listening to the arpeggios, but think the short ding-ring-out, or the *-stab sounds will work better in context. It would be good to cue people they overshot by making sure the special success sound is heard even if you zoom by it moving fast, and I think that will be harder to convey/interpret with an arpeggio.

@zepumph - Can you add the ding-ring-out as an option to the success sound list (feel free to remove any you think aren't being useful anymore from that list).

Subtle difference in sound when above/below success point.

• I think we should discuss this today. I thought of another reason this would be helpful, beyond what @jbphet has brought up before...if you're using the keyboard you're moving in 1-gridline increments by default. You could overshoot the success point and still hear the same staccato sound rate - essentially meaning two sequential keyboard presses give you the same sound feedback even if you've moved from just above/below to just below/above the success point. Would be good to have above/below be slightly different.

Balancing pickup/drop sounds

• For a little bit later in the sound design process...we should probably have the pickup/drop sounds be more in the background when you're also going to hear staccato ratio feedback sounds, but foregrounded more when you won't hear staccato ratio feedback sounds.

I also have some additional thoughts on tolerance (I think we should try having a 4-gridline total range of feedback across the board). I'll put these comments in #14.

[jbphet]

Thoughts:

• The sound design is really coming along!
• The choir sound, IMO, is awesome.
• The current "Single Arpeggiated Chord" option is good, but too quick in my opinion. I'd prefer the glock-marimba-c-maj-arp from above, though a little bit faster than it is and with less of a tail.
• The major and major 7th chords (not arpeggios) also feel too quick to me.
• I feel like the "success" sound should play every time the user moves through the correct proportion, even if they do so quickly. This will make them think, "Oh, there's something there that matters." This will be especially important for non-visual users.
• All of the sounds fade out entirely when not used, which gives the sound design a bit of an unnatural feel. IMO, the sound generators should be designed such that their sounds are always allowed to finish once initiated, but they stop initiating them when the aren't needed.
• When moving the markers with the keyboard, this cutting off of the sounds is particularly noticeable when moving quickly.
• The tremolo success sound was hard to get on the current master version, not sure why. I don't think it matters that much though, since most of the other success sounds seem superior.
• It's possible when using the keyboard to get the markers set up in such a way that it's hard to get to "full success", especially if using some mouse combined with some keyboard (I can demo at the meeting). Should we consider having the (non-shift) arrow keys move to the next tick mark instead of always a full increment?
• When playing with this on a touch screen, I found the sounds generated while moving both markers to be a bit overwhelming, since all of them tended to sound at various times. I think we should have a bit more hysteresis for the in/out behavior, and once the user is moving and keeping things green, they should get the choir and that's all.
• Is there something being designed for the visual portion to match the "super-success" state where the choir sings (i.e. where the user is moving both markers and maintaining the proportion)? If not, might I suggest something like a glowing yellow border?
• I'm not sure if this is an issue or not, but I noticed it while testing and thought we should make sure we want the behavior this way. Due, I suspect, to the nature of ratios, the feedback about whether being close to the correct ratio is asymmetrical on the linear grid that we present. For example, if "Challenge 1" is selected (1:2) and the left marker is at 2, sound will be produced if the right marker is at 5 (1 too many), but not at 3 (1 too few).