w3c / mnx

Music Notation CG next-generation music markup proposal.
176 stars 19 forks source link

Add support for multi-bar rests #284

Closed adrianholovaty closed 2 years ago

adrianholovaty commented 2 years ago

Here are a few proposals for how MNX can handle multi-bar rests. As you'll see, there are some open questions around where this data should live and how much flexibility MNX should provide.

We'll use the following music in each example. It consists of a full score, plus two individual parts: mnx_score_parts

Some things to note, before I get into proposals:

Here is the basic MNX document for this notation data, excluding any multi-bar rest information:

<mnx>
  <global>
    <measure-global>
      <directions-global>
        <time signature="4/4"/>
      </directions-global>
    </measure-global>
    <measure-global/>
    <measure-global/>
    <measure-global/>
    <measure-global/>
    <measure-global/>
    <measure-global/>
  </global>
  <part id="PartA">
    <measure>
      <directions-part>
        <clef sign="G" line="2"/>
      </directions-part>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <note pitch="E5"/>
        </event>
      </sequence>
    </measure>
  </part>
  <part id="PartB">
    <measure>
      <directions-part>
        <clef sign="G" line="2"/>
      </directions-part>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/4">
          <note pitch="D5"/>
        </event>
        <event value="/2">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/4">
          <note pitch="D5"/>
        </event>
        <event value="/2">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
  </part>
  <score name="Full score">
    <page>
      <system measure="1"></system>
      <system measure="5"></system>
    </page>
  </score>
  <score name="Part A">
    <page>
      <system measure="1"></system>
    </page>
  </score>
  <score name="Part B">
    <page>
      <system measure="1"></system>
    </page>
  </score>
</mnx>

Note that there are three <score> elements to represent the three different views of the music — "Full score", "Part A" and "Part B".

With that preamble out of the way, here are the proposals...

1. Put multi-bar rests in <part-layout>

Here, we invent a <multibar-rest> element, as a child of the existing <part-layout> element (see here). This puts the multi-bar rest information solely within our existing layouts system.

  <layouts>
    <system-layout id="PartALayout">
      <staff-layout>
        <part-layout part="PartA">
          <multibar-rest measure="3" length="2" />
        </part-layout>
      </staff-layout>
    </system-layout>
    <system-layout id="PartBLayout">
      <staff-layout>
        <part-layout part="PartB">
          <multibar-rest measure="1" length="2" />
          <multibar-rest measure="5" length="2" />
        </part-layout>
      </staff-layout>
    </system-layout>
  </layouts>
  <score name="Full score">
    <page>
      <system measure="1"></system>
      <system measure="5"></system>
    </page>
  </score>
  <score name="Part A">
    <page>
      <system measure="1" layout="PartALayout"></system>
    </page>
  </score>
  <score name="Part B">
    <page>
      <system measure="1" layout="PartBLayout"></system>
    </page>
  </score>

Using the power and flexibility of our layouts system means that you can create any kind of multi-bar rest you want for any <part>, and the data is all nicely contained in the <layouts> section of the document.

The downside is the complexity and verbosity. Any multi-bar rest would effectively need its own <system-layout>, which would get verbose quickly and would go against the spirit of making layouts reusable.

2. Put multi-bar rests in <system>

With that in mind, could we put the multi-bar rest data within <score> in a way that doesn't require us to use a layout?

In this second proposal, we invent two elements, <system-part> and <multibar-rest>. The idea is that <system-part> contains semantic data about a specific <part> in a specific <system>; this is distinct from layout data.

  <score name="Full score">
    <page>
      <system measure="1"></system>
      <system measure="5"></system>
    </page>
  </score>
  <score name="Part A">
    <page>
      <system measure="1">
        <system-part part="PartA">
          <multibar-rest measure="3" length="2" />
        </system-part>
      </system>
    </page>
  </score>
  <score name="Part B">
    <page>
      <system measure="1">
        <system-part part="PartB">
          <multibar-rest measure="1" length="2" />
          <multibar-rest measure="5" length="2" />
        </system-part>
      </system>
    </page>
  </score>

With this approach, we don't need to create layouts simply for the purpose of encoding multi-bar rests. It also gives us a firm place to attach other part-specific information of this kind (though I can't think of any examples...).

The downside is that the concepts of "part-specific semantic data" and "part-specific layout data" are very conceptually similar, to the point where it feels a bit obtuse to make a distinction. I could envision this being a point of confusion for MNX users: "Why is some of the information in <system-part> while other information is in <part-layout>?"

3. Put multi-bar rests in <measure>

What if we allowed ourselves to add multi-bar rest data directly within <part>? At first thought this might seem like it's mixing semantics with presentation, because <part> is intended to be "pure" semantic notation data — but the locations of multi-bar rests are semantic.

Here, <measure> elements within <part> get a multibar-rest attribute, whose value is the length of the multi-bar rest that begins in that bar. By default, this data is ignored. But a <part-layout> can opt in to using this data via multibar-rests="on".

  <part id="PartA">
    <measure>
      <directions-part>
        <clef sign="G" line="2"/>
      </directions-part>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure multibar-rest="2">
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <note pitch="E5"/>
        </event>
      </sequence>
    </measure>
  </part>
  <part id="PartB">
    <measure multibar-rest="2">
      <directions-part>
        <clef sign="G" line="2"/>
      </directions-part>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/4">
          <note pitch="D5"/>
        </event>
        <event value="/2">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/4">
          <note pitch="D5"/>
        </event>
        <event value="/2">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
    <measure multibar-rest="2">
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
  </part>
  <layouts>
    <system-layout id="PartALayout">
      <staff-layout>
        <part-layout part="PartA" multibar-rests="on" />
      </staff-layout>
    </system-layout>
    <system-layout id="PartBLayout">
      <staff-layout>
        <part-layout part="PartB" multibar-rests="on" />
      </staff-layout>
    </system-layout>
  </layouts>
  <score name="Full score">
    <page>
      <system measure="1"></system>
      <system measure="5"></system>
    </page>
  </score>
  <score name="Part A">
    <page>
      <system measure="1" layout="PartALayout"></system>
    </page>
  </score>
  <score name="Part B">
    <page>
      <system measure="1" layout="PartBLayout"></system>
    </page>
  </score>

This has a certain elegance (and I'm partial to it because the Soundslice data format works this way).

The downside is the lack of flexibility. For each <part>, there can only be one arrangement of multi-bar rests. So it would be impossible to create multiple <score>s with each having different locations of multi-bar rests for the same <part>.

Is this a dealbreaker? How common is it for the same underlying notation data to have different configurations of multi-bar rests in different contexts? I have the feeling that's a rare situation, but I'd like to get feedback from people in the trenches.

4. Put multi-bar rests in <measure> but make them addressable

Here's an attempt to solve idea 3's lack of flexibility. Instead of a boolean "multi-bar rests are either available or not for this particular <measure>", a multi-bar rest is given a class attribute. Then a layout can "opt into" any multi-bar rests of a certain class.

  <part id="PartA">
    <measure>
      <directions-part>
        <clef sign="G" line="2"/>
      </directions-part>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <multibar-rest length="2" class="multibars" />
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="C5"/>
        </event>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/2">
          <note pitch="G5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <note pitch="E5"/>
        </event>
      </sequence>
    </measure>
  </part>
  <part id="PartB">
    <measure>
      <multibar-rest length="2" class="multibars" />
      <directions-part>
        <clef sign="G" line="2"/>
      </directions-part>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/4">
          <note pitch="D5"/>
        </event>
        <event value="/2">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/4">
          <note pitch="E5"/>
        </event>
        <event value="/4">
          <note pitch="D5"/>
        </event>
        <event value="/2">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
    <measure>
      <multibar-rest length="2" class="multibars" />
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <rest/>
        </event>
      </sequence>
    </measure>
    <measure>
      <sequence>
        <event value="/1">
          <note pitch="C5"/>
        </event>
      </sequence>
    </measure>
  </part>
  <layouts>
    <system-layout id="PartALayout">
      <staff-layout>
        <part-layout part="PartA" multibar-rest-selector=".multibars" />
      </staff-layout>
    </system-layout>
    <system-layout id="PartBLayout">
      <staff-layout>
        <part-layout part="PartB" multibar-rest-selector=".multibars" />
      </staff-layout>
    </system-layout>
  </layouts>
  <score name="Full score">
    <page>
      <system measure="1"></system>
      <system measure="5"></system>
    </page>
  </score>
  <score name="Part A">
    <page>
      <system measure="1" layout="PartALayout"></system>
    </page>
  </score>
  <score name="Part B">
    <page>
      <system measure="1" layout="PartBLayout"></system>
    </page>
  </score>

This feels a bit harebrained to me to be honest, but I'm just exploring the problem space. Perhaps there's something here worth salvaging.

Concluding questions

Do any of these ideas resonate with you? Other proposals are also welcome!

And I'm especially interested in people's answers to the question I posed in idea 3. Is it good enough to limit each <part> to a specific set of multi-bar rest locations, or do we need to offer more flexibility?

Finally, if any developers of music-notation software want to share the details of their data model for multi-bar rests — what's worked and hasn't worked — that would be very valuable.

adrianholovaty commented 2 years ago

Related: See #121 for prior discussion on multi-bar rests, and see #34 for the larger/meta issue about differences between scores and parts.

clnoel commented 2 years ago

I notice that none of these match my proposal for multimeasure rests in #121(comment) and I was wondering why. It was already tuned for our recent changes in #185 except for including all the layouts in a <layouts> element. It most closely resembles option 2, but does not require a <system-part> element. This makes the multibar-rest a sibling of system-layout-change, which makes sense to me.

Obviously, I'm advocating for keeping the multi-measure-rest info in the <score>, or maybe <layout>, for the simple reason that multi-measure rests are not associated with a part, they are associated with a staff.

Here's my proposal, based on your example, rather than the one we were using in #121.

As an aside: Please note that in all your examples, we will need <system-layout> for both the Part A and Part B scores, regardless of which method we need for multi-measure rests, since the only default layout we have involves putting all parts in order on a staff (as in the Full Score score).

Option 2A:

<layouts>
    <system-layout id="PartAAlone">
        <staff-layout>
             <part-layout part="PartA"/>
        </staff-layout>
    </system-layout>
    <system-layout id="PartBAlone">
        <staff-layout>
             <part-layout part="PartB"/>
        </staff-layout>
    </system-layout>
</layouts>
<score name="Full score">
    <page>
        <system measure="1"></system>
         <system measure="5"></system>
    </page>
</score>
<score name="Part A" layout="PartAAlone">
    <page>
        <system measure="1">
            <multibar-rest staff="1" start="3:0" end="4:4/4" relative-width="1" label=2/>
        </system>
    </page>
</score>
<score name="Part B" layout="PartBAlone">
    <page>
        <system measure="1">
            <multibar-rest staff="1" start="1:0" end="2:4/4" relative-width="1" label=2/>
            <multibar-rest staff="1" start="3:0" end="5:4/4" relative-width="1" label=2/>
        </system>
    </page>
</score>

Notes on the attributes:


--Christina

dspreadbury commented 2 years ago

Following our discussion in our co-chairs' meeting today, a few thoughts from me:

My preference of these four proposals would be for option 1. Option 2 is a little redundant (there's no particular reason to tie the multi-bar rests to a system specifically, since it seems fine to encode them based on their starting bar number and the number of bars included within, which really has no relevance either to the system or to the page).

Options 3 and 4 feel wrong to me because multi-bar rests by their nature have to take into consideration the totality of the instruments/staves that are visible in the layout. Consider the case that you want to show a part layout for, say, violins 1 and 2 together (commonly done in film/commercial music so that the conductor can make balance changes on the fly from the stand). If the multi-bar rests for voilin 1 and 2 are encoded in <measure>, the consuming app has to parse out the multi-bar rests from the sequences for both parts, and work out where they can actually appear – it might be that the multi-bar rests in each part don't overlap at all, in which case none of them can be used, or that they partially overlap, in which case you would technically need to be able to split the encoded multi-bar rests into segments: only violin 1, only violin 2, and both violin 1 and 2.

So this feels wrong to me, since by encoding the information in the <measure> the consuming app can't actually just go ahead and follow the encoding to determine the correct appearance.

This, I think, means that the only practical approach is to encode multi-bar rests in layouts, since only layouts know which parts are going to be included, and can therefore encode the multi-bar rest ranges correctly.

I take slight issue with @clnoel's assertion that multi-bar rests are unambiguously at the level of staves. If you have a multi-staff instrument, such as a piano, you would never see a multi-bar rest in the right-hand staff but not in the left-hand staff, for example. And, indeed, as my example above where you have multiple instruments in the same layout demonstrates, they cannot even be correctly encoded at the level of an instrument (or part, in MNX parlance), because you may have multiple instruments in the same layout, and the multi-bar rests must take into account the music belonging to all of the instruments in the layout.

So I think it is in fact correct to say that multi-bar rests do live at the layout level, and that they apply to all staves in the layout, regardless of whether they belong to the same or different instruments. I would therefore propose that we follow option 1.

clnoel commented 2 years ago

I will take your word, @dspreadbury , that the only time multibar rests are used are when all instruments visible on a particular system are resting, or at least that that's the only case we want to cover right now. In this case, my option 2A becomes even easier, because you don't have to specify which staff is doing the multi-measure rest because they all are. We should not need a new layout, as in option 1, because we are not changing which parts are on which staves.

cecilios commented 2 years ago

My preferences:

clnoel commented 2 years ago

Thus far, the <layouts> element has only contained vertical information. That is, information that affects how many staves there are, how they are grouped,, and stemming information. On the other hand, <score> indicates horizontal information, such as how many measures are in a system, (that is, how many will fit horizontally on a page).

It looks like I might be outvoted, but I strongly believe that a multibar rest is a piece of horizontal information that belongs in the <score>.

@cecilios Parsers with no paging that care about how multiple parts combine on one stave (such as whether they are chorded or split-stem) at different times are still going to have to parse the page/system information in order to extract that information under our current spec, as this will be the most common usage of the <system-layout-change> element. Under 2A, <multibar-rest> is a sibling of that element.

cecilios commented 2 years ago

@clnoel Probably I overlooked the need to parse the page/system information. My main concern is not about parsing but about exporting pages and systems information. In MusicXML it is not mandatory to export that information, you can inform about it (<supports> elements), you can include multi-measure rests, and consumer applications are free to add page and system breaks as convenient and to render multi-measure rests as multi-measure or as many measures.

I would like to avoid that when adding support for MNX in such applications they will be forced to having to export page and systems breaks, perhaps fictitious.

Nevertheless, I do not understand why to tie multi-measure-rest to systems. A fifth alternative, in line with your proposal, would be to create a specific block for this information. It is perhaps the simplest approach that satisfies all objectives :

<score name="Part A" layout="PartAAlone">
    <multibar-rests>
        <multibar-rest staff="1" start="3:0" end="4:4/4" .../>
        <multibar-rest staff="1" start="14:0" end="17:4/4" .../>
        <multibar-rest staff="1" start="23:0" end="30:4/4" .../>
        ...
    </multibar-rests>
    <page>
        <system>
            // multibar-rest never goes here
        </system>
        ...
    </page>
    ...
</score>

Edited: To fix the <supports> elements markup

clnoel commented 2 years ago

You know what, I could get behind that. Let's call it option 2B.

adrianholovaty commented 2 years ago

Oooh, I like that last proposal from @cecilios. Create a <multibar-rests> section within <score>, separate from all the page/system/stave/layout stuff. Nice and simple!

It also means you can define multi-bar rests without needing to define <page>s. The <page> element is already optional, and it felt a bit dirty to define a page just to have a multi-bar rest.

I just have two suggestions/questions:

So with those two things in mind, I'd amend the @cecilios proposal to look like this:

  <layouts>
    <system-layout id="PartAAlone">
      <staff-layout>
         <part-layout part="PartA" />
      </staff-layout>
    </system-layout>
    <system-layout id="PartBAlone">
      <staff-layout>
         <part-layout part="PartB" />
      </staff-layout>
    </system-layout>
  </layouts>
  <score name="Full score">
    <page>
      <system measure="1"></system>
      <system measure="5"></system>
    </page>
  </score>
  <score name="Part A" layout="PartAAlone">
    <multibar-rests>
      <multibar-rest start="3" duration="2" />
    </multibar-rests>
  </score>
  <score name="Part B" layout="PartBAlone">
    <multibar-rests>
      <multibar-rest start="1" duration="2" />
      <multibar-rest start="5" duration="2" />
    </multibar-rests>
  </score>

This includes the <system-layout> sections as pointed out by @clnoel (thanks for the catch!).

adrianholovaty commented 2 years ago

An update — @mdgood has suggested that we call the element <multimeasure-rest> given that we use the term <measure> in MNX (not bar).

cecilios commented 2 years ago

@adrianholovaty I agree with your proposed changes. Staff is not needed and I don't see the need to use measures locations as the whole measure is a rest. I copied the <multibar-rest> from another example without paying attention to its attributes. And also I agree with @mdgood proposal of using <multimeasure-rest>, it is more coherent.

clnoel commented 2 years ago

Do we need a label that is different from duration? It would be optional, of course, defaulting to the duration. Other than that, good tweaks @adrianholovaty. I approve.

Details/Extra Info: I'm okay with using measure number, rather than measure location. I think the original proposal used measure location in order to match <system-layout-change>.

I'm thoroughly behind dropping the staff attribute. I was going to mention it myself, then I forgot. Sorry. We can certainly revisit an optional staff attribute here, but I don't see that being needed for v1.

I am pleased about switching to <multimeasure-rest>, as that is actually closer to what I call them in my head (and in my original proposal back in #121.

adrianholovaty commented 2 years ago

@cecilios and @clnoel — thanks for the quick reactions! OK, I will go ahead and put a pull request together with this approach.

@clnoel Regarding a label attribute, yes, that makes sense (sorry, I meant to write this when you suggested it in the previous comment). I'll add that in the pull request as well.

adrianholovaty commented 2 years ago

OK, commit 8e4c1fb added multimeasure rest support to the spec!

Note that issue #286 is still open — regarding a way to encode multimeasure rest style (old-style vs. new-style).