search

w3c / mnx

Music Notation CG next-generation music markup proposal.
175 stars 19 forks source link

Synchronization demo application (v2) and the issues it raises #132

Closed notator closed 6 years ago

notator commented 6 years ago

I'm trying to re-join #67 as quickly as possible, but don't want to overload it unnecessarily, and think it would be helpful if we could first take a look at at least some of the issues listed below. These issues have arisen while writing the new version of my synchronisation demo application (v2) as promised in #95. The previous version (v1) was mentioned in https://github.com/w3c/mnx/issues/67#issuecomment-362202972.

This issue just provides some background information about (v2) and the issues it raises. I'll raise them as separate issues, as soon as I get some feedback:

  1. New Issue: I think MNX's <performance-...> elements should be renamed <playback-...>. We should differentiate more clearly between live performances (by performers), synchronisation with the playback of external files (mp3s etc.) and the playback of information contained inside MNX files. The latter would be better described as playback, not performance information.
  2. For #66: MNX durations should be in whole-note units (as they currently are), but I still think that timings in live performances, synchronisation with the playback of external files, and playback using MNX-internal information can only be done in "real time" using absolute units (seconds).
  3. New issue: The <performance-event> start attribute is redundant. The origin for playback event times is the playback's start time, and the start-time of each event is the end-time of the previous event in the same part. So the <performance-event> start attribute is always ignored. Deleting it would be a useful simplification (see e.g. Example 36 in §5.2.13.1). It is only necessary for MNX to define the <performance-event> duration attribute.
  4. New issue: The MNX <performance-event> (or <playback-event>) element could contain an element that defines MIDI information.
  5. For #99: (v2) just demonstrates how a cursor can be made to jump around the score ad lib. It would be great if we could make some progress with MNX's strategy for dealing with repeats, da Capo etc.. I have a few ideas, but obviously its not an easy subject and lots of people have been giving it some thought. It must be made possible for synchronising applications to give their users options like "play bars 13-56, ignoring repeats" or "play bars 1-32 including all repeats but not the dal Segno" etc.

@joeberkovitz said in https://github.com/w3c/mnx/issues/67#issuecomment-374615828

First, let's acknowledge that we should never assume that all applications will use SVG as a graphical engine.

Agreed. Both (v1) and (v2) use SVG scores, but that is only for demonstration purposes. The object of this exercise is to investigate how information in an MNX file is transferred to an instantiation and then used.

what the app does

Both (v1) and (v2) synchronize the same external mp3 recordings, but (v2) now also

  1. (silently) plays the durations that would have been defined inside an MNX file. An <event>'s absolute, calculated playback duration (seconds) depends on the tempo and either on the <performance-event> duration, the <event> duration, or the <event> value (in that order). Unfortunately, there is no MNX file behind (v2), so this app always uses the default event durations derived from <event> value. It is, however, clear that any <event> duration and/or <performance-event> duration information in the MNX would be consumed by the instantiating application: The <event> duration information would affect the layout, but the <performance-event> duration would not.
  2. demonstrates how cursors can jump about in scores to cope with repeated sections. There are no repeat, da Capo or dal Segno signs in (v2)'s score, and there is no MNX file containing such information, so I have just created a simulation.

how it works

(v1) used a copy of the score stored in the MNX repository. This was raw, unstructured SVG created by an Adobe plug-in. I susequently created a new file by re-structuring the original, using a proper SVG <defs> section, converting all the (hypothetical) MNX <event>s to <g class="event">s, and including all the MNX information in a special namespace (which I named cwmn). Since this re-structured file contains <g class="event">s, I was able to use it as the basis for the score for (v2). Note that (v2)'s score is as re-flowable as its predecessor.

The only functional change in (v2)'s score, is that I have now added a cwmn:simX attribute to each <g class="event> in the file. The cwmn:simX value is the alignment point of the sim to which the event symbol belongs. Both this value and the event's system index would be known to any instantiating application that spaces sims across systems. (Note that whole-bar rests have the cwmn:simX value of the first sim in the measure.) The score is organised "part-wise", so (v2) uses the cwmn:simX values both as x-alignments and as "sim-IDs" when creating the cursor trajectory definitions (a list of [time, line coordinates] pairs) from the cursor JSON files. Note that the JSON files for the silent "Metronome" and "Repeats Demo" options could have been completely deduced from the information in the score. I decided against that for simplicity.

The new application's component files can be found on GitHub at https://github.com/notator/altMNX/tree/master/cursors/demos/verticalLineCursorDemo Try it out here http://james-ingram.de/altMNX/cursors/demos/verticalLineCursorDemo/verticalLineCursorDemo.html

joeberkovitz commented 6 years ago

Feedback:

New Issue: I think MNX's elements should be renamed .

If you think so, open a new issue and see where it goes. My feeling is that either name works, and that we should avoid spending time debating terminology unless there's a shared sense in the CG that it's important.

For #67 (@notator you said #66, but I think that may have been a typ): MNX durations should be in whole-note units (as they currently are), but I still think that timings in live performances, synchronisation with the playback of external files, and playback using MNX-internal information can only be done in "real time" using absolute units (seconds).

In the proposal floated for #67 performance sync points are specified in absolute units of seconds so this is not an issue.

New issue: The start attribute is redundant.

It is not redundant because silence may exist between performance events, which, being non-notational in nature, always consist of sound production. We do not need a performance event to represent the absence of a sound, as there is no metrical framework in a performance-event description of music.

New issue: The MNX (or ) element could contain an element that defines MIDI information.

I think we have been over this territory of using MIDI in the group already, and the co-chairs are not in favor of using a low-level description like MIDI, which is geared to old-school concepts like channels, banks, program changes, controllers, etc.

Closing since this issue combines 5 separable issues into a single item.

Note that demos need to be used in support of a proposed specification, they are not issues in themselves.