ryaanahmed
commented
2 years ago Show ephemerality of the gesture by animating the canvas instead of having a static representation.
Map each gesture to an audio buffer or envelope that gets applied to another signal.
Gestures
Assessment
The “Music” instrument generated from the tool creates a cool musical tune given the input in the canvas. It is pretty effective at creating a tune that seems to match the drawing.
(After writing this I just found the Show Axis button—is this “off” by default intentionally?)
A couple nitpicks:
Besides the Music generated instrument, the others (Pad, Slider Looper, Spatial Instrument, and Step Sequencer) don’t seem to find their place. It seems as though there is a single pitch generated for each stroke in the input that is used for all of these instruments. While this isn’t very interesting, especially given the potential complexity of the input, the output is often a high-pitched sine tone that is very shrill and hurts my ears. I think these instruments should either be removed or repurposed in some way.
Overall, the interface is pretty easy to use and understand.
Ideas
The first change to make the experience more pleasing is to make the generated tones lower in pitch.
Like the colors module, I think that there are other musical attributes that we can draw on to map to features in the input, such as dynamics, timbre, and harmony. I have yet to dig into the code to see how the input is featurized but it might be worthwhile exploring this. Additionally, since we allow for multiple strokes, it could be interesting to include white space as something that is incorporated into the final product—perhaps the space or shape between strokes could affect the duration of a tone or introduce a rest to the music.
The current implementation creates a concatenation of notes, though a stroke in the drawing is continuous. Users (myself included) might at first expect the drawings to create a continuous sound of some kind. This might be a limitation of the existing API, in which case we should add this functionality if we want to implement it.
There is a correspondence to the order of the stroke input and the order of the generated notes. I think this should be reflected somehow in the canvas. Perhaps small numbers next to each stroke, or a color-coding system (though this could be confusing in that it may imply to the user that the color denotes something other than note order, especially given another tool of ours is color-based).
An interesting idea that Ryaan brought up was perhaps being able to draw an envelope using this tool. Can we build another instrument that could take advantage of this?