Updating the design of the music interaction

[GoogleCodeExporter]

In as much detail as possible, please describe what you would like to see.

1. Adding notes by clicking on the music staff
-When you hover over a the staff, a 'ghost' note appears and follows your 
mouse, then becomes a 'placed' note once you click. These existing notes can be 
dragged to new locations and deleted as well.

2. Reworking the 'Play current sequence' button.
-The button can be into a play icon that's closer to the music staff (maybe 
right above the treble clef)

3. Reordering the remaining buttons
-The 'submit' button should be the farthest-most button on the bottom right, 
and the 'clear staff' button can go directly to the left of the submit button. 
The 'Play current sequence button can be visually separated and placed on the 
bottom-left.

I've attached a screenshot of what this might look like. It shows a scenario in 
which two notes have been added to the staff, and a third note is about to be 
added because the mouse is hovering over that part of the staff.

Here are a few other thoughts as well, some of which is wishful thinking :)
-It would be nice if the notes lit up while your sequence was playing.
-It could be nice to add the option in the editor to place measures on the 
music staff and toggle between treble and bass clef.
-It could also be nice to have toggles in the reader view to change between 
adding whole notes, half notes, quarter notes, etc.
-The 'Play Target Sequence' button is a bit awkward, but I don't have any good 
ideas on how to make it better.

What operating system are you using?
Mac OSX

Please provide any additional information below.

Original issue reported on code.google.com by amitdeut...@google.com on 31 Jan 2015 at 3:13

Attachments:

• MusicWidget.png

[GoogleCodeExporter]

(De-assigning Mike for the time being until he has time for this; he's 
currently working on the 'share exploration' functionality.)

Mike, just to give context about where this came from: we're about to finish 
building the redesigned learner view, so Amit was doing an audit of the various 
interactions and trying to make them look really nice!

Btw, just to record a thought I had about "play target sequence" -- I think 
that, at some point, we should actually remove this from the music widget, and 
make it into a separate noninteractive "play audio" widget, since this 
functionality really belongs in the 'content' part of the state and isn't 
really part of the interaction itself.

Also, +Xinyu for info, since she's currently looking into creating some 
music-based explorations.

Original comment by s...@seanlip.org on 31 Jan 2015 at 4:17

• Changed title: Updating the design of the music interaction

[GoogleCodeExporter]

Good point, thanks for de-assigning.

Re: the non-interactive "play audio" widget, I realized that this might be 
inconvenient for the authors, since you will then need to create the target 
audio sequence twice: once when you're configuring the music widget and 
determining what the correct answer is, and once when you're adding the 
noninteractive play audio widget to the state content.

Original comment by amitdeut...@google.com on 2 Feb 2015 at 4:21

[GoogleCodeExporter]

I'd love to get Michael's/Xinyu's thoughts on this too, but I think splitting 
off the "play expected audio" part would make the interaction more flexible, if 
anything. If we do that then it opens up possibilities like "play the bass line 
of this audio clip" or "identify the first two notes of this tune". Right now 
it's a bit like having a TextInput interaction that can be used only for 
questions of the form "Type in the string X".

Original comment by s...@google.com on 2 Feb 2015 at 4:38

[GoogleCodeExporter]

I definitely like the idea of splitting the play functionality off into a 
seperate widget, as it would surely increase the possibilities for learner 
interactions. I am concerned that, in this context, a "Play audio" widget might 
not necessarily work with the music widget becuase there isn't an audio file 
being played directly but actually a series of midi notes triggering audio 
files. Surely there are ways around this, just thought it was worth mentioning. 
 Ultimately, I think the resulting functionality and extensibility increase 
would justify the refactoring it would take to make this "play nice".

Original comment by wagnerdm...@gmail.com on 2 Feb 2015 at 5:18

[GoogleCodeExporter]

All good points. Also, there might be music interactions beyond just 'recreate 
the following target music sequence', so separating the music playing into a 
separate noninteractive widget would free up the interactive music widget to be 
more flexible.

Original comment by amitdeut...@google.com on 2 Feb 2015 at 5:25

[GoogleCodeExporter]

I agree with Sean and Mike. There are many cases that require both a 
demonstration and an answer with the music widget (and often they will contain 
different notes). It seems like a good idea to put the 'demonstration' part in 
the 'content' part of the state rather than the 'answer' part. 
A question: by "play audio" widget, are we referring to something that plays an 
audio clip, or something like the current music widget, which can play a 
sequence of notes that are displayed on a score to the learner (and highlight 
parts that are being played etc)? (Or maybe both?)

Original comment by wxy.xi...@gmail.com on 2 Feb 2015 at 5:26

[GoogleCodeExporter]

OK, it sounds like the consensus is towards splitting.

To Xinyu's last question -- that's a good distinction. I think we are really 
describing two RTE extensions. One of them simply plays an audio file (and this 
seems to be a common request: see 
https://code.google.com/p/oppia/issues/detail?id=159 ). The other plays a 
sequence of MIDI notes, perhaps combined with an associated interactive score 
display.

Original comment by s...@seanlip.org on 2 Feb 2015 at 6:36

[GoogleCodeExporter]

I've filed issue 568; this and issue 159 are the issues for tracking the 
subsidiary RTE extensions. So, the scope of this issue (566) is now for 
improving the design of the current music interaction.

Thanks very much for the quick feedback!

Original comment by s...@seanlip.org on 2 Feb 2015 at 8:14