Should global (system) notations be interspersed within regular parts?

[joeberkovitz]

Instead of placing within <measure> elements within <global>, should system notations be placed in individual parts and tagged in some way, perhaps via a scope="global" attribute or something similar?

This would avoid the problem of the <global> or similar element being very sparse with most measures containing nothing.

However, this approach could scatter system notations throughout the score in a happenstance way, since there is no particular part that makes sense for them to belong to.

While they could be forced to be placed in the first part, this leads to ambiguities as sometimes the first part is not visible on all systems. In MusicXML, the result is that system notations tend to appear in the topmost visible part, which varies unpredictably from score to score.

[joeberkovitz]

Chair discussion: this question also relates to part groups, which could share notations across their member parts. Would we encode this sharing also even though it's not score-wide? (e.g. Don Giovanni minuet).

[cschardt]

Why not have both? Appearing in <system> is brief and convenient. Appearing in the parts gives the opportunity to put individual displacements per part to each notation. How could this be achieved otherwise?

[jsutdolph]

I don't understand what the system element is. I have searched the document at https://w3c.github.io/mnx/specification/ and there is no system element described

[mscuthbert]

I have no problem with sparse usage especially if the encoding is optional. I think for musical analysis purposes it is much better to have everything that affects a measure of one part (whether it affects that measure alone or the whole system) to be either in the data for the part/measure itself or some other special purpose element and not in another part’s data hidden among information that only pertains to the other part.

[joeberkovitz]

Note that we changed the name of <system> to <global> in the initial spec draft -- see https://w3c.github.io/mnx/specification/#the-global-element

[fablau]

I second what @cschardt and @mscuthbert have proposed above, I don't see any other alternative at the moment. It is definitively convenient to have system notation references in separate parts. Unless we create a separate, "master" file which includes just system notation to which parts can always reference to.

[jsutdolph]

The trouble with global as described in the spec. is that this would not be easily readable by eye. You would need to count all those empty measure elements to ascertain whether the key change is (say) in bar 232. It would be better just to include non-empty measure elements and give them a number.

[jsutdolph]

Apropos cschardt 'why not have both' . I am very much against including redundant copies of the same information in a file. What does the client program do if the copies of the information are not in agreement? Is it forced to check this and handle it and make a decision? Do we have to define which is correct? An acceptable alternative would be to include the copied information in a comment. What about having a single global element for all parts at the very start of the cwmn section.

[cecilios]

To answer this question, we need to clarify and agree on the purpose of the global element.

1. If the global element represents just common content (i.e. to facilitate not having to repeat things on every part) then it should not be mandatory to place common content in the global element. But then this common content must be repeated in each part.

2. But if it represents global constrains then only things that can not be changed in each part should be placed there. Common content not representing a constrain should not be allowed to be placed in global.

   If the global element represents constrains my answer to 'why not have both'? is: The idea of allowing global notations to be placed in individual parts and tagged in some way, perhaps via a scope="system" attribute would be redundant with the idea of having the global element. Having both is having alternative ways of encoding the same fact. As one of the objectives of MNX was to give clear directions on how to encode things and to avoid alternatives, having both approaches should not be allowed.

   The problem of the global element representing global constrains is that the common barlines must be always included (as is currently proposed, although we can change the decision), as this will force to include a lot of empty measures elements. It would be better just to include common barlines optional and require to include only measure elements containing global changes (time signatures, keys, etc.) and reference them by number or other reference. This could also facilitate dealing with multi-metric scores, as suggested in #55.

[joeberkovitz]

(Note: I'll switch from saying "system" to "global" to avoid further confusion over the name change.)

@cecilios: The intended purpose, which we could change, was for <global> to represent common content that can not be overridden by parts. The idea of encoding the same thing in two different ways (e.g. allowing global notations to occur within parts) goes against two of the main architectural goals of MNX: a) to eliminate alternative ways of doing the same thing, and b) to represent any particular piece of information once, in a single document location.

Note that I'm excluding cases like multi-metric scores, or scores with non-corresponding key signatures in different parts. (As you pointed out in #68, the very presence of <global> implies that certain classes of scores, e.g. multi-metric, cannot be represented -- and in these cases, one cannot meaningfully employ <global>.)

@cschardt @fablau: So, while it is undoubtedly convenient for some purposes to have global notations within parts, at the same time it's inconvenient in at least two ways that seem important:

• It creates many ways for documents to encode content with internal conflicts, which must all be checked.
• It requires a consumer implementation to scan all parts up front, in order to discover all global notations that may be scattered throughout them. This virtually guarantees a two-pass scan of the entire document.

In order to answer these objections, I think there needs to be a compelling case showing that embedding global notations in parts is more than just a convenience for a subset of applications.

@jsutdolph: I think you offer a good argument in favor of numbering.

[joeberkovitz]

I'll inject a new point here to consider, and I'm not sure yet how I feel about it. It relates to #55 as well as this issue.

It's been pointed out that <global> is not useful in the case of scores that are polymetric or polytonal. However, such scores often consist of one set of parts in one meter(key), and another set of parts in another meter(key). In these cases, would it make sense to be able to describe the constraints that apply to each subset of parts? To do this, we could generalize the concept of <global> to permit it to apply to any subset of parts, not only to all parts.

Since this would hugely expand the universe of possibilities that an application needed to deal with, I would expect it to be excluded from the standard profile. The question of the standard profile's definition is not resolved and is reflected in #68, by the way.

[mscuthbert]

Does the concept of <global> really become something like <part-group-settings> with a default part-group as applying to the entire score?

[cecilios]

@joeberkovitz

The intended purpose, which we could change, was for to represent common content that can not be overridden by parts.

I fully support this. This implies that it is not mandatory to include the common content. Therefore, including empty measure elements is optional. In this case, It should be considered not to allow to include empty measure elements as they do not add anything but noise. In this case, all measure elements included in global must be referenced by number or other.

would it make sense to be able to describe the constraints that apply to each subset of parts? To do this, we could generalize the concept of to permit it to apply to any subset of parts, not only to all parts.

My first impression is that this is a very interesting idea for dealing with polymetric and polytonal scores, that should be explored. Need to define a simple way for defining the subsets of parts. Perhaps overlapped sets should be allowed, for defining the content common to all parts, but this could create inconsistencies between them. Probably it would be much better to allow only disjoined sets at the expense of having to repeat the all parts common content in each global set.

this would hugely expand the universe of possibilities that an application needed to deal with

Not sure what are you referring to. An app. will first have to read the global element to get the common content and then use it for filling the gaps when reading each part. The only difference between having a global element or a few (probably not more than two or three) is splitting the global content in several tables indexed by part id. Not complex neither expensive.

[fablau]

@cschardt @fablau: So, while it is undoubtedly convenient for some purposes to have global notations within parts, at the same time it's inconvenient in at least two ways that seem important: 1. It creates many ways for documents to encode content with internal conflicts, which must all be checked. 2. It requires a consumer implementation to scan all parts up front, in order to discover all global notations that may be scattered throughout them. This virtually guarantees a two-pass scan of the entire document. In order to answer these objections, I think there needs to be a compelling case showing that embedding global notations in parts is more than just a convenience for a subset of applications.

@joeberkovitz I understand that, and of course that could be a problem. I am wondering then if there is a way to have something like a "global" or "master" file which can be used as reference for those global signs.

[bhamblok]

I think this is a very important discussion. Please allow me for a suggestion to think about for a couple of minutes. It's probably going to raise a new issue with a lot more questions, but... it might provide a solution for all of next issues:

• the current issue #2 with global notation elements
• issue #32: "distinguish directions that are not scoped to any particular sequence"
• the polymetric issue #55 (#68)
• "support for system flow and break control" issue #57
• the "number of staves for part" issue #43
• a lot more issues to come

How does a musician/conductor read his music? He won't start to read part 1, finish it completely, then "backup" (~cf. musicXML) to the beginning of the piece to start to read part 2 to start over again for the next, and the next part... Written music has evolved for hundreds of years and came to a form which is able to contain everything(?) we dreamed of. Why now change the order of information it contains?

Then again, I believe a (human) musician isn't reading in a strict "time-wise" sequence either, but rather in small chuncks: part-wise, yes, but rearly any further than a couple of beats.

So I'm questioning if the "part-wise" format is the most ideal solution! A more "human-approach" could work as well.

I would like to introduce the concept of a "sub-measure". Other element-naming suggestions could be "chunk" / "fraction" / "division" / please help me out here.

Here is a (very brief!) example-implementation:

<score>
  <cwmnx profile="standard">
    <measure-fraction number="1">
      <directions>... global directions ...</directions> (cf. issue #2)
      <part pid="1">
        <directions>... part directions ....</directions> (cf. issue #32)
        <sequence staff="1">...</sequence>
        <barline> (a polymetric barline cf. issue #55)
      </part>
      <part pid="2">
        <directions>... part directions ...</directions> (cf. issue #32)
        <sequence staff="1">...</sequence>
        <sequence staff="2">
          <directions>... staff/voice/sequence directions ...</directions>
        </sequence>
      </part>
    </measure-fraction>
    <measure-fraction number="1.x">
      <part pid="1">
        <sequence staff="1">...</sequence>
      </part>
      <part pid="2">
        <directions>...</directions> (cf. issue #32)
        <sequence staff="1">...</sequence>
        <sequence staff="2">...</sequence>
      </part>
      <barline> (a full-measure barline cf. issue #55)
    </measure-fraction>
    <measure-fraction number="2" system="new"> (cf. issue #57)
      <directions>... some new global directions ...</directions> (cf. issue #2)
      <part pid="1"> ... </part>
      <part pid="2"> ... </part>
      <part pid="x"> (cf. issue #43)
        <sequence staff="3">...</sequence> (cf. issue #43)
      </part>
    </measure-fraction>
  </cwmnx>
</score>

While I was writing this down, I think we simply can keep the element-naming for a "sub-measure" to simply be: "measure". Because in 99.x% of all cases, there won't be polymetry. But I want to keep the example above as is to point out a "measure" does not need to be a "full" measure.

This approach won't need a two-pass scan, but "mimics" the way we "as humans" read a paper piece of sheet music. I would like to emphasize the importance of the hundreds of years of evolution in sheet music. Back then, people already realized music is time-wise (sort of speak), not part-wise. So, ..., why change the order of information now as it has been evolving for so many years?

[joeberkovitz]

@fablau: Please help me understand why the <global> element is not already exactly the same as the "master file" that you are proposing. While it is not a separate file, it contains exactly the information that you suggest, as far as I can tell, and there's no real need to have multiple files describing a score.

@bhamblok: For what it's worth, I have given a lot of thought to abandoning the partwise format (and this is not to dismiss any fresh thinking like yours). When all measures are in step (i.e. no polymetric music), this yields a reasonable structure with no need for fractions. In fact, MEI uses just such an approach. But with polymeter, it all falls apart. The use of measure fractions strikes me as giving up something very big in the name of a "human-like" principle that doesn't yield big benefits. The thing we give up is a direct and simple mapping between measures in the score and measures in the schema. In its place, developers would face a lot of complexity in fragmenting measures when writing CWMNX, and reassembling them when reading it, and music-reading humans are not the people who will be looking at an MNX file! My feeling to date has been that musical scores are a two-dimensional structure, of a kind that hierarchies are ill suited to represent. Neither the partwise nor timewise approaches are perfect, and we have to pick the least worst one :-)

[bhamblok]

@joeberkovitz : As you mention, when all measures are in step (i.e. no polymetric music), a timewise format yields a reasonable structure. Which could solve many more issues than we are creating while maintaining a partwise format. However, when polymeter does occur, a whole lot of complexity (as a developer), regardless of the timewise format is added as such, while the direct and simple mapping between measures in the score, and measures in the schema is lost anyway. As well for the musician, as for the developer.

So I would still like to discuss the topic of a timewise format a bit further. We could keep a measure schema in most music (non polymetric music). While in others, we could approach the schema like the image below. Each "measure" (-fraction) takes the number of beats, up until some part reaches a "breakpoint"/barline.

original picture taken from the Lilypond blog: http://lilypondblog.org/2014/05/independent-meters

[joeberkovitz]

After thinking about this a while longer, I began to ask myself whether there were other ways to look at this problem besides MusicXML's timewise or partwise options, or @bhamblok's suggestion of breaking measures up into little pieces. I think I have found at least one such alternative. While I'm not convinced it's the best solution yet, it's worth including in the discussion. It certainly has a different set of pros and cons, and perhaps has some appeal compared to the other avenues.

The basic approach is to identify subsets of parts that share an identical measure structure, and bundle these beneath a single element whose structure is somewhat "timewise". (For a non-polymetric score, there is only one such element, so the typical case is not complicated by this approach.) Within this element, the information we have been calling "global" is encoded once, and shared by all parts in the subset.

It's easiest to begin with an example of how this works for a non-polymetric score. Here, the new element <measures> encodes a stream of measures whose structure is shared by some set of parts. Each <measure> element now bundles up the former "global" information, plus one or more child elements describing that measure's content for each part. The <global> element thus disappears, and its contents migrate to the new hierarchy under <measure>.

<cwmnx profile="standard">
  <!-- set of measures for some parts that share a measure structure -->
  <measures>
    <!-- define all the parts that share this structure -->
    <part id="p1" style="y: above;">
      <part-name>Voice</part-name>
      <instrument-sound>voice.aa</instrument-sound>
    </part>
    <part id="p2" style="measure-numbering: none;">
      <part-name>Piano</part-name>
      <instrument-sound>keyboard.piano</instrument-sound>
    </part>

    <!-- Measure 1 in all parts -->
    <measure>
      <!-- first, directions common to this measure in all parts -->
      <directions>
        <tempo bpm="60" value="/4"/>
        <time signature="3/4"/>
        <key fifths="-3" mode="major"/>
        <instruction class="tempo-indication">Andantino</instruction>
      </directions>

      <!-- next, <part-measure> elements for each part with content in this measure -->
      <part-measure part="p1">
        <!-- directions for this measure applying only to the vocal part -->
        <directions>
          <staves number="1"/>
          <clef line="2" sign="G"/>
        </directions>
        <!-- content of this measure for the vocal part -->
        <sequence>
          <event measure="yes">
            <rest/>
          </event>
        </sequence>
      </part-measure>

      <part-measure part="p2">
        <!-- again, piano part-specific directions for this measure -->
        <directions>
          <staves number="2"/>
          <clef line="2" sign="G" staff="1"/>
          <clef line="4" sign="F" staff="2"/>
        </directions>
        <sequence staff="1">...</sequence>
        <sequence staff="2">...</sequence>
      </part-measure>
    </measure>

    <!-- Measure 2 in all parts -->
    <measure>
      <!-- (If there were directions common to both parts, they'd occur here) -->
      <part-measure part="p1">
        <directions>...</directions>
        <sequence>...</sequence>
      </part-measure>

      <part-measure part="p2">
        <directions>...</directions>
        <sequence staff="1">...</sequence>
        <sequence staff="2">...</sequence>
      </part-measure>
    </measure>
  </measures>
</cwmnx>

The approach to encoding a polymetric score is very straightforward: such a score includes multiple <measures> elements, one for each set of parts with the same metrical structure. In extreme polymetry where each part has a different structure, there would only be one <part> in each <measures>.

Pros:

• No need to correlate the <global> measure elements with the corresponding <part> measure elements. All of this information is now nested within a single <measure> element, and these occur sequentially.
• No empty <measure> elements; sparse encoding comes naturally to this structure. If there are no global directions in some measure, just leave them out. If a part has no music for some <measure>, then its <part-measure> can be omitted.

Cons:

• To parse the information for only a single part, consumers must process a filtered list of <part-measure> elements that only apply to that part.

[bhamblok]

@joeberkovitz: nice thinking, I like it :-) However, in polymetry, unfortunately there is another "con" in your proposal: where/how would we indicate system-brakes / page-flows?

[cecilios]

@joeberkovitz and @bhamblok : I like very musch your proposals.

The latest proposal from joeberkovitz does not solve the problem of encoding common directions for all parts (i.e. page breaks or system breaks) for polymetric music.

The approach suggested by bhamblok of using measure-fractions is much better to find a unified solution for all score types: normal scores (single time), polymetric and no-metric scores. The bias to preserving the measure element without any supporting use cases may be the root of current problems. By using measure-fractions:

• For normal music (a single common time signature) the measure-fraction is the measure (measure preserved for most music).

• For polymetric music, the measure-fraction could be something similar to what bhamblok proposed (I would suggest not splitting beamed groups and ties), but page break, system breaks and other notations common to all parts can be encoded only once for all parts.

• And for no-time music, measure-fractions are producer defined, and they will suggest feasible break points in all parts. But in any case page break and system breaks can also be encoded only once for all parts.

The measure-fraction elements provide feasible break points, so this simplifies line breaking algorithms for music not based on measures and could be and advantage for free flow layouts. Current algorithms must first identify suitable places for breaks an classify them by 'goodness' (i.e. for polymetric, better choose places with barline in all parts).

[joeberkovitz]

The co-chairs discussed this issue today at length. We recommend that the discussion focus on comparing the two alternatives that we feel are most viable:

1. Require the <measure> element to carry a number attribute which must sequentially number all measures in the score. This makes the correlation between measures in <global> and <part> elements clear, and allows empty measure elements to be omitted for sparse representation. At the same time, allow <global> elements to be provided for specific subsets of parts, thus supporting polymetric scores (perhaps outside the standard CWMN profile).

2. Reorganize both global measure content and part-based measure content as children of a single <measure> element as described here. This sacrifices placing all part-related content under a <part>, in order to to place all similarly-structured measure content under a <measure>.

In both of the above cases, directions that are shared by all staves of the music can be flagged as such via a specific attributes. This is only an issue for polymetric content, and we are not sure what sorts of directions would even be shared in such scores given the differences in measure layout, but it feels like more of an edge case than a central concern driving the design of the encoding.

We do not feel the following approaches are viable:

3. Including global content in part-specific measures. This is duplicative and violates the parsimonious-design principle of MNX.

4. Organizing polymetric scores in terms of measure fractions. Our objections include:

   1. This does not work for scores in which different parts progress at different tempos.
   2. Architecturally, it's bad to complicate an encoding without serious gains from doing so. System and line break positions could still be specified at any desired location without fragmenting measures, and common directions can be supported as described above.
   3. An event may be split across measure fractions, in which case it is not easy to tell from looking at the fraction what is going on inside it -- or, the representation of events gets subdivided, which is really overcomplicating things.
   4. Small, local changes in content (e.g. rebarring one part of a polymetric score) can force the entire score to be globally re-fragmented in a different way. This hinders use cases MC4: comparison and DEV3: dynamic score generation.

[mscuthbert]

Wonderful. Can I suggest that when numbering elements MNX distinguishes consistently between something like <measure sequence="12"> which means the 12th (or 13th if we decide to do 0-indexed sequence) and something like <measure number="12"> which means the measure that a musician would consider measure 12 (pickup measures, 1st/2nd endings, a score fragment beginning in media res etc. may interfere with this). It's possible that we might often see things like <measure sequence="11" number="234">

[mdgood]

In MusicXML 3.1 we use number for the identifier and text for the displayed measure number, but that is an artifact of the number attribute doing double duty in earlier MusicXML versions. Something like sequence and number sounds good to me.

[joeberkovitz]

@mscuthbert Yes, that's an excellent improvement (since this would be indeed a sequence number and not a musically-oriented number).

[bhamblok]

Great, I agree with all arguments 4. i. ii. iii. and iv. here above. My initial proposal would indeed not be the best solution to cover these arguments.

But I do have some issues with some global definitions. In my opinion "part-names" and "instrument-sounds" are not global definitions. How many times does it occur a flute alternates with the piccolo in the same part? I think we should try to avoid to globally define "things" which, like on a paper piece of sheet music, could alter at any time. That makes @joeberkovitz's example here above also a bit verbose. Let me try to alter this by example, starting from @joeberkovitz's proposal:

<cwmnx profile="standard">
  <!-- set of measures for some parts that share a measure structure -->
  <measures>
    <!-- Measure 1 in all parts -->
    <measure>
      <!-- first, directions common to this measure in all parts -->
      <directions>
        <tempo bpm="60" value="/4"/>
        <time signature="3/4"/>
        <key fifths="-3" mode="major"/>
        <instruction class="tempo-indication">Andantino</instruction>
      </directions>

      <!-- next, <part> elements for each part with content in this measure -->
      <part id="p1" style="y: above;">
        <part-name>Flute</part-name>
        <instrument-sound>wind.flute</instrument-sound>
        <!-- directions for this measure applying only to the vocal part -->
        <directions>
          <staves number="1"/>
          <clef line="2" sign="G"/>
        </directions>
        <!-- content of this measure for the vocal part -->
        <sequence>
          <event measure="yes">
            <rest/>
          </event>
        </sequence>
      </part>

      <part id="p2" style="measure-numbering: none;">
        <part-name>Piano</part-name>
        <instrument-sound>keyboard.piano</instrument-sound>
        <!-- again, piano part-specific directions for this measure -->
        <directions>
          <staves number="2"/>
          <clef line="2" sign="G" staff="1"/>
          <clef line="4" sign="F" staff="2"/>
        </directions>
        <sequence staff="1">...</sequence>
        <sequence staff="2">...</sequence>
      </part>
    </measure>

    <!-- Measure 2 in all parts -->
    <measure>
      <!-- (If there were directions common to both parts, they'd occur here) -->
      <part id="p1">
        <part-name>Piccolo</part-name>
        <instrument-sound>wind.piccolo</instrument-sound>
        <directions>...</directions>
        <sequence>...</sequence>
      </part>

      <part id="p2">
        <directions>...</directions>
        <sequence staff="1">...</sequence>
        <sequence staff="2">...</sequence>
      </part>
    </measure>
  </measures>
</cwmnx>

I think those "global" definitions (including style-attributes) should be inherited from one measure to the next, just like the <clef> element will need to be inherited for any specific staff (and part) from one measure (or beat) to the next. Then again: <part-groups> would be some kind of a global definition... Not sure yet where we could define that. Probably as a direct child of the <measures> element (in combination with a ref-id to another <measures> element in case of polymeter?).

[notator]

@joeberkovitz said

After thinking about this a while longer, I began to ask myself whether there were other ways to look at this problem besides MusicXML's timewise or partwise options, or @bhamblok's suggestion of breaking measures up into little pieces. I think I have found at least one such alternative. While I'm not convinced it's the best solution yet, it's worth including in the discussion. It certainly has a different set of pros and cons, and perhaps has some appeal compared to the other avenues.

In the same spirit, I'd like to add the following alternative to the discussion:

A basic requirement is that there be system breaks in any notation, so I'd like to propose using the system (rather than the page) as the point at which the part-wise option breaks. This makes larger chunks of part-wise data than in @bhamblok's proposal, but smaller than in FaurReveSample-cwmnx.xml (i.e. MusicXML's part-wise option). This would also allow multi-metric parts - which all have to break at the end of the same system. The container hierarchy would then be:

<page>
    <system>
        <part>...</part>
        <part>...</part>
        <!-- etc. (more parts) -->
    </system>
    <system>
        <!-- parts -->
    </system>
    <!-- etc. (more systems) -->
</page>

This proposal would mean that measures would have to be able to break at system breaks, but not inside systems. Maybe that leads to something simpler than either @bhamblok's original proposal or @joeberkovitz' suggestion above.

This suggestion is related to issues raised in FaurReveSample.cwmn.svg. If the .cwmn.svg format is adopted, then each system has to have somewhere where its basic elements (stafflines, clefs, key signatures, instrument names etc.) are written. It would therefore be convenient if <system> were part of the container hierarchy. Issues relating to <global> and <directions> are also raised in FaurReveSample.cwmn.svg, so I think we need to resolve #25 before we can finally resolve this issue (#2).

[bhamblok]

@notator: sorry, I cannot agree because we would lose the ability to reflow the score (cf. #DEV5, #MP5 and #RLP2)

[notator]

@bhamblok I dont see how you would lose the ability to reflow the score.

Parsers could easily concatenate <part> elements in successive <system>s (and pages), so as to create an abstract list of parts that can be reflowed into a new score, with new system (and page) breaks.

[bhamblok]

@notator: yes but what is the point of encoding systems when a parser generates it's own/new systems out of a concatenation of parts from the original encoded systems.

A system also isn't a "natural" breakpoint for music. It's a necessary "evil" :-) because our sheets of paper used to ran out of space. Therefor, a measure is a logical and "natural" breakpoint because it divides our music in groups of beats with the same meter.

[notator]

@bhamblok

what is the point of encoding systems when a parser generates it's own/new systems out of a concatenation of parts from the original encoded systems.

The point, of course, is to be able to re-flow the score! :-) Note that in this case, the parser would be parsing semantic information that includes whether the measure is complete, or whether it continues in the next system. A .cwmn.svg file contains semantic information that is exactly equivalent to the semantic information in a <cwmnx> element. So the only difference between this proposal and the situation in @joeberkovitz' original FaurReveSample-cwmnx.xml file, is that the parts would break at the ends of systems. That's very close to the original MusicXML part-wise definition, so should not be too difficult to understand or implement.

A system also isn't a "natural" breakpoint for music. It's a necessary "evil" :-) because our sheets of paper used to ran out of space.

System breaks may not have anything to do with musical form, but they are a fundamental part of score rendering, both on paper and on screens (neither of which ever have unlimited space). The same can be said of page breaks -- page turns are also a "necessary evil", on screen too. :-).

Actually, systems and pages are related not only to the limited sizes of screens and pieces of paper. They have a lot to do with the way we read. Reading happens in chunks of information. Music that is presented in one long, continuous, scrolling system is harder to read and memorize. Music notation is much easier to read, comprehend and memorize, if we have the enhanced graphical information that is provided by systems and pages. One shouldn't underestimate their contribution to orientation: "Go to page 5, measure 2". System and page breaks can, of course, be ignored if one is only interested in the musical form.

[bhamblok]

@notator: wouldn't it be nice if we could render the music, in a reflowable way, without having to parse and reformat the source-content? The <system>-element like you propose so, is a container which encapsulates content which is by nature NOT to be captured. I'm very much aware of the importance of orientation on a graphical representation of sheet music. There has to be some way to indicate system-breaks. But I also believe the semantics of a "system" is not to encapsulate content, but rather to indicate a "line break".

[joeberkovitz]

@notator: Let's not confuse the issues here. CWMNX must permit optional system and page break information to be layered on top of semantic data, and this requirement is captured in issue #57 which will be taken up soon. As @bhamblok said, SVG does far more than simply capture the location of system/page breaks, and this problem rightly belongs to issue #25.

[notator]

I'd also like not to confuse the issues, its just that my proposal here resulted from looking closely at your code in the FaurReveSample-cwmnx.xml file.

Can you say why the optional system and page break information must be layered on top of the semantic data, and what that is going to mean for the semantic data itself? Could you give an example of how you think the solution is going to look? Maybe in #57. Maybe #57 should be folded into this issue?

[mdgood]

For the same reason that the co-chairs all object to #25. CWMNX is a semantic format for conventional Western music notation, and it makes no architectural sense to us to have graphical aspects be primary in a semantic format. Let us please restrict that discussion to issue #25.

[clnoel]

@notator I feel that document flow and layout (system & page breaks, system indent, measure width, vertical spacing between systems, etc.) is a separate issue from this one and should be addressed separately in #57 . We spend time and effort making sure that our documents will print right (with no text collisions, orphan measures, or unreadably spaced systems) and we want to be able to specify all those details when sharing our files. However, the ability to optionally put in such visual features (that will be ignored if the document is reflowed by the recipient) is separate from the importance of being able to put in the semantic system-wide directions.

I approve of the idea of having measure "number" (printed value), separate from the "sequence" value. Here's an important question, though: Should the "sequence" value be a document internal index, that has no musically semantic value, and is just used as a reference between parts, or should we make it a "time-wise" index, in which case a measure in a repeated section is allowed to have more than one "sequence" index? I know we haven't worked out how to specify repeats yet, and this concept of a sequence index might help us out.

Also, Allowing multiple sequence indices might give us a good way of specifying directions that only apply on one repeat ("2nd time 8va") by (optionally) associating them with one of the measure's sequence indices.

In regards to whether to use Option 1 or 2, I would rather use Option 1 and stick to part-wise scores. There are too many elements that are cross-measure (ties, slurs, etc.), and tracking all of them for all parts at once seems like way more of a headache when parsing the document than single part-at-a-time. It will be much easier to parse a sparsely populated global element, duplicating the data into each part internally, and then come back and fill in each part's individual information.

--Christina Noel Musicnotes

[jsutdolph]

1. Pros

   • Has the advantage of being similar to MusicXML which we have a lot of experience with
   • Simple handling of barlines at different times in different parts (a rare case but the consensus is that it should be possible to represent it)
   • Extraction of individual parts is almost trivial
   • [Is perhaps a more 'natural' structure for performers?]

2. Cons

   • Difficulty of handling global directions (common)

3. Pros

   • An elegant solution for handling global directions (common)
   • [Is perhaps a more 'natural' format for composers and conductors?]

4. Cons

   • Complicated handling of barlines at different times in different parts (rare)
   • Makes the measure an even more important element, where we already face the criticism of it being too important so that we have problems encoding a significant amount of music (I think this was a thread in a previous discussion)

I am not sure that the 'natural' format argument is a very important one.

[jsutdolph]

I am wondering if global directions, although attractive have a complexity cost which is quite out of proportion to their value. The MusicXML pattern of defining everything separately for each part seems so much simpler, and it handles all cases.

[bhamblok]

@clnoel: I believe tracking cross-measure elements (ties, slurs, etc.) could be even more (performance- and memory-) efficient if you actually don't parse them for all parts at once. That is of course if we decide to use ref-id's like in the initial spec draft, and not like we are used to in the musicXML spec using a "start" and "stop"-element.

@jsutdolph:

• MusicXML had some shortcomings which we are trying to overcome here, so although it worked well for a lot of cases, referring to an advantage of being similar to MusicXML might not be a "pro" in some arguments.
• Extraction of individual parts should and could always be trivial (using XPath). In that context, I believe some features in music analysis software could also become a bit easier when we migrate to a more time-wise structure, like eg. harmony-analysis.

I would like to point out again that written music has evolved for so many years using a time-wise structure, in which a measure (or at least a number of "moments in time") is written down at once. Which works! As @Kalvos said in issue #25:

(30 years ago, because of hardware expense) we all learn to read so-called computer-compatible digits. That taught me the lesson about circumscribing approaches to present perspectives, limitations, or expense.

Nowadays, our hardware (and software) are able to approach problems in a more "human" way. So let's prepare for the further future and approach written music in the same way as it always has been, like physical audio-waveforms moving through our physical air, which is (and will always be ...) time-wise.

[notator]

@bhamblok

Nowadays, our hardware (and software) are able to approach problems in a more "human" way. So let's prepare for the further future and approach written music in the same way as it always has been...

Yes, I agree completely with that. But the problem here is that music notation uses event symbols (#60) that are ordered both horizontally and vertically. Event symbols are not just like the one-dimensional audio wave that reaches our ears. Analog piano-roll notation may be okay for mechanical devices, but it can't compete with notations that use event symbols when it comes to ("human") legibility.

[bhamblok]

@notator I don't agree. Audio waves are more than a one-dimensional stream, let's not discuss that into detail over here. But I believe we are here trying to discuss the best possibilities of capturing CWMN in a digital format, and I also believe CWMN comes close to describe "how musicians should be able to regenerate these audio-waveforms as the composer intended" :-) To include graphical notations where the notation itself becomes the piece of art, is (if i'm correct) out of scope of CWMN. I hope we can end up with a format where we can blend all these different kinds of notations together, starting with the two-dimensional event-symbols (vertical: parts/staves/pitches, horizontal: time/rhythm), ... ending up with other kinds of notations where in the notation itself a standard doesn't even exists yet.

[clnoel]

This looks like it is turning into a general discussion of whether we want to do part-wise or time-wise scores. I'm pretty sure that the arguments I've seen here are why MusicXML supports both. I'm not sure we're going to get a true community consensus about which way we want to do this. No matter which way we pick, someone is going to be unhappy.

That being said, @bhamblok, even with ref ids, I would much rather track ties and slurs part-wise. About the only thing that is more difficult to track when doing part-wise scores is what happens when the part isn't printed for a couple of systems and the tempo or meter changes during that time. This is especially true if we are allowing different parts to have different meters, which it looks like we want to.

That said, you have a good point about the analysis use case, which isn't one of mine, so I forgot it. How many other of our use cases have a CLEAR computational advantage one way or the other?

Christina Noel Musicnotes

[joeberkovitz]

These choices are not exactly the same as MusicXML's partwise and timewise approaches. To be clear, in both of the options put forward, ties and slurs would be tracked partwise. This is because even in the mostly-timewise option 2, any given measure's part-specific content is segregated for each part within a <part-measure> element. In option 2, <part-measure> elements for a given part, in successive <measure>s, contain the same music as the sequence of <measure>s in a partwise score.

That said, it's clear that no option is perfect. It's been very useful to have these different points of view surfaced. The co-chairs will review all the input and select a definite approach next week, then post it here and turn it into a pull request.

[clnoel]

@joeberkovitz The choices ARE exactly the same as MusicXML's partwise and timewise approaches, since the (simplified) schemas in MusicXML 3.0 are:

<score-timewise>
    <measure>
        <part/>
        <part/>
    </measure>
    <measure>
        <part/>
        <part/>
    </measure>
</score-timewise>

<score-partwise>
    <part>
        <measure/>
        <measure/>
    </part>
    <part>
        <measure/>
        <measure/>
    </part>
</score-partwise>

I don't see a conceptual difference between that and our two current choices. It's just that we're now making a distinct naming difference by calling the timewise parts "part-measure"s instead.

Of course, now that I'm looking at it, even option 2 doesn't hit what @bhamblok was talking about with needing to do harmony analysis, since it's NOT truly timewise. More like measure-wise. It's closer, but it's not there. The only way I can think of to actually hit easy harmony analysis is to start doing beat or sub-beat slices, which I think we decided wasn't going to work right back when we were talking measure-fractions.

That rambled a bit, but I think I just reaffirmed my opinion that partwise (Option 1) is better.

Christina Noel Musicnotes

[bhamblok]

@clnoel is right to say a "measure-wise" scheme is also not the ideal scheme to do "harmony analysis" (which btw isn't my goal for using mnx, but I needed an example where a more, not a truly-, timewise approach would be appropriate).

Anyway, however I'm aware of the fact that the existence of the timewise spec in musicXML wasn't used in the field, I really would like to be able using CWMNX in a NON-partwise scheme. So whatever we choose, we should be able to easily and fully convert one to the other, using XSLT, or even making another profile for it?

On top of that, I stand by my opinion we should transcribe written sheet music one on one to a (new) digital format as it evolved for hundreds of years (which is "measure-wise" in CWMN). Keeping the order of data, the lexicon (like "voices" instead of "layers" cf. some musicXML-discussion in the past), and most importantly incognizant of existing software tools.

[joeberkovitz]

@clnoel Sorry, I thought mistakenly that you felt a timewise format couldn't accommodate per-part tracking of slurs and ties. (The difference that I was referring to between CWMNX/option 2 and MusicXML/timewise, is just the ability to include global or cross-part content in the scope of the <measure> element, but above the scope of the <part> or <part-measure> element).

I think it's fair to say that the bias is towards Option 1 right now among the chairs because it has so much prior usage, preserves part continuity, and there is no perfect (or "natural") mapping scheme from an inherently two-dimensional structure to a linear XML hierarchy.

[joeberkovitz]

Recording the chairs discussion of this issue yesterday: we will proceed with a PR that embodies so-called Option 1. Recapping from above:

Require the <measure> element to carry a sequence attribute which must sequentially number all measures in the score. This makes the correlation between measures in and elements clear, and allows empty measure elements to be omitted for sparse representation. At the same time, allow more than one<global> element to be provided, one for each subset of parts sharing a common structure, thus supporting polymetric scores (perhaps outside the standard CWMN profile).

[mscuthbert]

Just to clarify -- is the "number attribute which must sequentially number all measures" to be called "sequence" not number, which would be reserved for the numbering as musicians use?

[joeberkovitz]

@mscuthbert my bad; I've edited the comment above to reflect the use of sequence as already agreed.

[joeberkovitz]

Note comments in #81: changed sequence to index to avoid confusion with the <sequence> element.

[joeberkovitz]

In the course of putting together PR #81 it became apparent that sparse encoding of measures could increase the effort for a consumer implementation to determine the actual makeup and length of the score. All parts must be decoded in full to discover the actual location of the last measure, since any part could always include <measure index="N"> where N is larger than any measure index seen so far.

Given this, do people have any second thoughts about sparse encoding? Placing back into Active Review to make sure.