Music sheet events

[jaredjj3]

Requirements

When the user interacts (click, tap, keyboard press during focus, etc.) with a rendered score, the score should emit events that can be hooked into. An event may or may not contain information about what was interacted with.

NOTE: Any events tied to a cursor should be emitted by the cursor instead. See #158.

Hints

Before developing anything, I highly recommend that you make a list of events + payloads (see MusicDisplay Event definitions below).

There is prior art in stringsync.

• EventBus: Pubsub mechanism for events with payloads.
• MusicDisplay Event definitions: A list of all the different events and their payloads built on top of osmd.
• SVGEventProxy: An interaction sink that translates DOM events to MusicDisplay events.
• State machine for the pointer: The SVGEventProxy pipes DOM events to this, and this is what ultimately calculates the MusicDisplay events.
• MusicDisplayLocator: Efficiently finds a osmd cursor snapshot given an (x, y) position.

[jaredjj3]

I'm starting to work on this now. In both approaches, vexml would expose the same API. I'm going with the Single event listener approach unless I run into a hard block.

Single event listener (preferred)

Install a single event listener on the <svg> or <canvas>. On interaction, vexml calculates the data needed to satisfy the API.

• ➕ Less event listeners means a lower memory footprint.
• ➕ Compatible with <svg> and <canvas> backends.
• ➕ Better custom handling for elements whose bounding boxes eclipse each other.
• ➕ Performance is constant with respect to the music sheet size (data structure memory may grow linearly, however).
• ➖ Calculating what elements are in scope would require a complex data structure, such as a quad tree.

Event listener per element

Install an event listener for each vexflow.Element in a <svg>. vexml would pipe and transform the data from the native events to conform to the API.

• ➕ There is low complexity transforming event data.
• ➕ Interactions can be faster since there's no calculation overhead.
• ➖ Performance scales linearly with respect to the music sheet size (?).
• ➖ <canvas> cannot be support with this approach.