Missing measure lines when converting to humdrum

[WolfgangDrescher]

I have this MusicXML file: 1-Beatus-vir.xml.zip

When I try to convert this file from MusicXML to Humdrum (Kern), measure numbers and measure data lines are missing from the output.

I'd be happy to contribute with a PR if I get a little guidance where to start for this issue.

Unrelated, but is it possible to pass options to the converter, e.g. to ignore stem direction signifiers (/ and \) during conversion.

[gregchapman-dev]

Thanks for the report! I'll take a look at this today to see what the issue might be. (Unrelatedly, there is no such option at the moment, although there is an option mechanism that could be extended in future to include such things: for example, converter21's MEI exporter has an option to request an MEI v4 file instead of the usual MEI v5 file.)

[WolfgangDrescher]

It looks like m21Stream.recurse().getElementsNotOfClass(m21.stream.Stream) will not return any music21.bar.Barline elements other than the last bar line.

https://github.com/gregchapman-dev/converter21/blob/9183277295758b43873e03aa42a77ff1441dc706/converter21/humdrum/measuredata.py#L281-L283

[gregchapman-dev]

I took a look at the barline issue. When music21's MusicXML parser (music21/musicxml/xmlToM21.py) imports this file, it produces almost no barlines in the music21 score (measure.rightBarline is None and measure.leftBarline is None). So converter21's Humdrum exporter doesn't produce any. This is weird: I need to think more about what this actually means. The MusicXML file doesn't actually mention any barlines beyond the very last "light-heavy" one. But should that be interpreted as "normal default barlines should be assumed"? And if so, is it music21's bug for not doing that? Or does music21 also require that interpretation, and it is converter21's bug for not assuming default barlines? And then there's the big "did Dorico produce incorrect MusicXML" possibility, in which the MusicXML should have explicit normal barlines? (This seems unlikely, since Musescore displays this MusicXML file with barlines, but I'll check.) I'm off to read some MusicXML/music21 documentation.

[gregchapman-dev]

One very quick answer from the MusicXML docs: "If a barline is other than a normal single barline, it should be represented by a element that describes it." So the MusicXML is doing the right thing, and normal barlines should be assumed.

[WolfgangDrescher]

Unrelatedly, there is no such option at the moment, although there is an option mechanism that could be extended in future to include such things: for example, converter21's MEI exporter has an option to request an MEI v4 file instead of the usual MEI v5 file.

This is probably not really important since for this specific option with stem directions it's quite easy to use the humdrum tool shed and parse it after the conversion.

[WolfgangDrescher]

But should that be interpreted as "normal default barlines should be assumed"? And if so, is it music21's bug for not doing that? Or does music21 also require that interpretation, and it is converter21's bug for not assuming default barlines?

It is difficult for me to help with these questions as I have not dealt with music21 before.

One very quick answer from the MusicXML docs: "If a barline is other than a normal single barline, it should be represented by a element that describes it." So the MusicXML is doing the right thing, and normal barlines should be assumed.

Okay still the question remains whether this element is missing in music21 (which I can hardly imagine, as such behavior would certainly have been noticed beforehand) or if we need to implement it into converter21 because Kern syntax explicitly needs them.

[gregchapman-dev]

I don't see anything conclusive in the music21 docs about what a missing barline means. I am leaning towards interpreting a missing barline in music21 the same way it should be interpreted in MusicXML, since music21 is pretty MusicXML-centric. I did try exporting to MusicXML from this music21 score, and it came out the way it went in, with all those barlines still missing, rather than present, with bar-style=none.

If I'm right, then converter21 has some work needed (in the Humdrum exporter, and maybe also in the MEI exporter), to interpret missing barlines as "normal" instead of as "not present".

I will try to get a conclusive answer from Michael or Jacob (the two main developers of music21). More likely Jacob, since Michael is on sabbatical.

[WolfgangDrescher]

Thank you very much for your efforts. This feature would be a great help for a project of a student of mine, as he feels more comfortable writing the scores in Dorico and then converting them to Humdrum.

[gregchapman-dev]

No worries! I filed an issue in music21 #1669 (and sent a private email to @jacobtylerwalls as well).

[craigsapp]

Is the goal is to have it import/export correctly into/from music21, or to convert to Humdrum? The latter can be done in VHV:

I am seeing barlines, which do not seem to have any strange format in the MusicXML data (Dorico does some weird things as I remember that I needed to compensate for in the MusicXML-to-Humdrum converter).

To use the same convert on the command-line, it is called musicxml2hum from humlib. (xml2hum is the prior version of the converter in Humdrum Extras which I keep for reference).

Here is the score imported and exported to/from Musescore, which is useful for testing:

Click to view MusicXML for score exported from Musescore

```xml 1. Beatus vir Partie 1 Orlando di Lasso MuseScore 4.1.1 2023-11-27 6.99911 40 1596.77 1233.87 85.7252 85.7252 85.7252 85.7252 85.7252 85.7252 85.7252 85.7252 title Partie 1 composer Orlando di Lasso Stimme 1 keyboard.piano.grand 1 1 78.7402 0 Stimme 2 keyboard.piano.grand 2 1 78.7402 0 97.56 -0.00 170.00 2 0 2 1 G 2 16 1 A 4 16 1 breve begin Be B -1 4 8 1 whole flat middle a A 4 8 1 whole end tus D 4 4 1 half up single vir. A 4 8 1 whole single qui A 4 4 1 half up single in C 5 6 1 half down begin sa B 4 2 1 quarter down A 4 2 1 quarter up G 4 2 1 quarter up A 4 4 1 half up 0.00 -0.00 114.52 E 4 4 1 half up middle pi C 5 4 1 half down middle en C 5 2 1 quarter down B 4 2 1 quarter down A 4 4 1 half up G 4 4 1 half up middle ti E 4 4 1 half up end a G 4 6 1 half up begin mo A 4 2 1 quarter up B 4 2 1 quarter up G 4 2 1 quarter up C 5 4 1 half down C 5 4 1 half down B 4 4 1 half down middle ra B 4 2 1 quarter down A 4 2 1 quarter up A 4 4 1 half up A 4 4 1 half up G 1 4 4 1 half sharp up middle bi 0.00 -0.00 114.52 A 4 16 1 breve end tur, 8 1 whole A 4 8 1 whole single et A 4 8 1 whole 4 1 half A 4 4 1 half up single qui C 5 4 1 half down single in A 4 4 1 half up begin ju E 4 2 1 quarter up middle sti F 4 2 1 quarter up G 4 2 1 quarter up A 4 2 1 quarter up G 4 6 1 half up F 4 1 1 eighth up begin E 4 1 1 eighth up end F 4 4 1 half up G 4 4 1 half up middle ti 0.00 -0.00 114.52 A 4 4 1 half up end a G 4 2 1 quarter up begin me A 4 2 1 quarter up B 4 2 1 quarter down C 5 2 1 quarter down D 5 4 1 half down D 5 2 1 quarter down C 5 1 1 eighth down begin B 4 1 1 eighth down end A 4 2 1 quarter up B 4 2 1 quarter down C 5 4 1 half down B 4 4 1 half down middle di A 4 6 1 half up middle ta A 4 2 1 quarter up middle bi G 4 8 1 whole end tur, 4 1 half A 4 8 1 whole single et D 4 4 1 half up single in 0.00 0.00 114.52 A 4 8 1 whole begin sen F 4 6 1 half up end su E 4 2 1 quarter up F 4 2 1 quarter up G 4 2 1 quarter up A 4 4 1 half up E 4 8 1 whole begin co F 1 4 4 1 half sharp up middle gi G 4 4 1 half up middle ta G 4 2 1 quarter up E 4 2 1 quarter up C 5 4 1 half down C 5 4 1 half down A 4 4 1 half up end bit 4 1 half A 4 4 1 half up begin cir B 4 4 1 half up middle cum C 5 8 1 whole middle spe B -1 4 4 1 half flat down middle cti 0.00 -0.00 70.00 A 4 8 1 whole middle o G 4 4 1 half up end nem A 4 4 1 half up begin De A 4 2 1 quarter up B 4 2 1 quarter down C 5 2 1 quarter down A 4 2 1 quarter up B 4 2 1 quarter down C 5 2 1 quarter down D 5 4 1 half down D 5 2 1 quarter down G 4 2 1 quarter up B -1 4 4 1 half flat up A 4 8 1 whole end i, 4 1 half F 1 4 4 1 half sharp up begin cir G 4 4 1 half up middle cum A 4 4 1 half up middle spe A 4 4 1 half up G 4 4 1 half up middle cti F 4 8 1 whole natural middle o 0.00 -0.00 305.78 E 4 4 1 half up end nem C 5 4 1 half down begin De C 5 2 1 quarter down B 4 2 1 quarter down A 4 2 1 quarter up G 4 2 1 quarter up A 4 2 1 quarter up E 4 2 1 quarter up F 4 2 1 quarter up G 4 2 1 quarter up A 4 2 1 quarter up F 4 2 1 quarter up G 4 2 1 quarter up A 4 2 1 quarter up G 4 2 1 quarter up F 4 2 1 quarter up F 4 2 1 quarter up E 4 1 1 eighth up begin D 4 1 1 eighth up end E 4 8 1 whole D 4 16 1 breve end i. D 4 16 1 breve light-heavy 62.62 2 0 2 1 G 2 D 5 16 1 breve begin Be C 1 5 8 1 whole sharp middle a D 5 8 1 whole end tus G 4 4 1 half up single vir, D 5 8 1 whole single qui D 5 4 1 half down single in F 5 6 1 half down begin sa E 5 2 1 quarter down D 5 2 1 quarter down C 5 2 1 quarter down D 5 4 1 half down A 4 4 1 half up middle pi F 5 4 1 half down middle en F 5 2 1 quarter down E 5 2 1 quarter down D 5 4 1 half down 60.58 C 5 4 1 half down middle ti A 4 4 1 half up end a C 5 6 1 half down begin mo D 5 2 1 quarter down E 5 2 1 quarter down C 5 2 1 quarter down G 5 4 1 half down C 5 2 1 quarter down middle ra D 5 2 1 quarter down E 5 2 1 quarter down C 5 2 1 quarter down D 5 2 1 quarter down E 5 2 1 quarter down A 4 2 1 quarter up B 4 2 1 quarter down C 5 2 1 quarter down A 4 2 1 quarter up D 5 2 1 quarter down E 5 2 1 quarter down D 5 2 1 quarter down C 5 2 1 quarter down C 5 2 1 quarter down B 4 1 1 eighth up begin A 4 1 1 eighth up end B 4 6 1 half up B 4 2 1 quarter up middle bi 59.42 A 4 8 1 whole end tur, 8 1 whole F 5 16 1 breve single et 4 1 half D 5 4 1 half down single qui F 5 4 1 half down single in D 5 4 1 half down begin ju A 4 2 1 quarter up middle sti B 4 2 1 quarter down C 5 2 1 quarter down D 5 2 1 quarter down C 5 6 1 half down B 4 1 1 eighth up begin A 4 1 1 eighth up end B 4 4 1 half down C 5 4 1 half down middle ti D 5 4 1 half down end a C 5 4 1 half down begin me 72.92 C 5 2 1 quarter down D 5 2 1 quarter down E 5 2 1 quarter down F 5 2 1 quarter down G 5 6 1 half down F 5 1 1 eighth down begin E 5 1 1 eighth down end D 5 2 1 quarter down E 5 2 1 quarter down F 5 4 1 half down E 5 4 1 half down middle di G 5 4 1 half down middle ta G 5 2 1 quarter down F 5 2 1 quarter down F 5 4 1 half down F 5 4 1 half down E 5 4 1 half down middle bi F 5 16 1 breve end tur, 71.71 4 1 half D 5 8 1 whole single et A 4 4 1 half up single in D 5 8 1 whole begin sen C 5 4 1 half down end su A 4 4 1 half up begin co A 4 4 1 half up B 4 4 1 half down middle gi C 5 6 1 half down middle ta A 4 2 1 quarter up F 5 8 1 whole D 5 8 1 whole end bit 4 1 half E 5 4 1 half down begin cir F 5 4 1 half down middle cum G 5 4 1 half down middle soe 122.31 G 5 4 1 half down F 5 4 1 half down middle cti E 5 4 1 half down middle o D 5 2 1 quarter down end nem E 5 2 1 quarter down F 5 2 1 quarter down D 5 2 1 quarter down E 5 2 1 quarter down F 5 2 1 quarter down G 5 4 1 half down G 4 2 1 quarter up begin De A 4 2 1 quarter up B 4 2 1 quarter down C 5 2 1 quarter down D 5 4 1 half down D 5 2 1 quarter down E 5 2 1 quarter down F 5 4 1 half down D 5 8 1 whole end i, 4 1 half C 1 5 4 1 half sharp down begin cir D 5 4 1 half down middle cum E 5 8 1 whole middle spe D 5 4 1 half down middle cti 122.34 C 5 4 1 half down middle o A 4 4 1 half up end nem F 5 6 1 half down begin De E 5 2 1 quarter down D 5 2 1 quarter down C 5 2 1 quarter down D 5 2 1 quarter down B 4 2 1 quarter down C 5 2 1 quarter down D 5 2 1 quarter down E 5 2 1 quarter down F 5 2 1 quarter down E 5 2 1 quarter down D 5 2 1 quarter down D 5 4 1 half down D 5 2 1 quarter down C 1 5 1 1 eighth sharp down begin B 4 1 1 eighth down end C 1 5 4 1 half down D 5 16 1 breve end i. D 5 16 1 breve light-heavy ```

[gregchapman-dev]

I agree with @craigsapp (and was about to mention this tool, although I probably would have confused xml2hum and musicxml2hum). The humdrum parsing/writing code in converter21 is generally translated from (a large subset of) Craig's humdrum code, so Craig's code supports more music notation features, and is definitely always faster, since it is in C++ rather than Python. The only thing I don't know is how musicxml2hum's MusicXML parser compares with music21's. I suspect they are similar.

[craigsapp]

I probably would have confused xml2hum and musicxml2hum

I should probably rename xml2hum to xml2hum-old :-)

I still use xml2hum to compare outputs from musicxml2hum and xml2hum when there are problems in musicxml2hum, but musicxml2hum handles more complex cases in general (such as grand-staff parts for piano music).

[WolfgangDrescher]

Thank you @craigsapp! I don't know how I missed that there is a humlib tool for this. I only found hum2xml, but probably I was in the wrong directory (humextra) or something. I will try it out!

[craigsapp]

You can also use musicxml2hum via verovio (since humlib is embedded in verovio):

verovio -f musicxml-hum -t humdrum file.musicxml

this will create file.krn which is a conversion of file.musicxml using musicxml2hum.

-f musicxml-hum means load from MusicXML via the musicxml2hum converter (rather than the musicxml-to-mei converter).

-t humdrum means output data to Humdrum.

---

Note that musicxml2hum has two options:

option	meaning
-s	Preserve stem directions (by default they are omitted).
-r	Include a **recip spine for the composite rhythm of the score (useful for avoiding having to calculate this information from the plain Humdrum score).

[WolfgangDrescher]

Works perfectly, thank you! Even figured bass numbers are imported!

[gregchapman-dev]

Most excellent!

BTW, I have heard from Jacob, and he confirms that no barline in music21 means the same thing as no <barline> in MusicXML. So I will be fixing that in the Humdrum (and maybe MEI) exporters in converter21 soon. Thanks for the report!

[craigsapp]

On the webpage https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/barline:

If a barline is other than a normal single barline, it should be represented by a element that describes it.

So if there is no <barline> for the end of the measure, it should be assumed to be an implicit regular barline:

      <barline location="right">
        <bar-style>regular</bar-style>
      </barline>

An invisible barline should be encoded explicitly:

      <barline location="right">
        <bar-style>none</bar-style>
      </barline>

[gregchapman-dev]

Fixed on develop branch (a pretty stable branch that previews stuff that's ready for the next release).