**harm parser integration in music21

[napulen]

I have made a parser of the **harm syntax for Roman Numeral Analysis. As a suggestion from Mark Gotham, I would like to integrate this parser within music21.

So far, the parser lives in this repository https://github.com/napulen/harmparser and consists in a single python file.

I would like to commit this parser on top of the latest music21 commit and make a PR.

I would like to ask in this issue two things:

• Does it make sense to have the **harm parser in music21?
• What is the best location/way of integrating it within the codebase of music21?

[mscuthbert]

Would you be willing to relicense this as BSD? If so I think that we can make it work with music21 -- probably I'd just take the HarmDefs which are INCREDIBLE and so useful and also the def parse() call -- from there it wouldn't be too hard to put it within the music21/humdrum/spineParser file as a separate tool.

[mscuthbert]

It could also still be licensed under the Apache license -- I believe that the licenses are compatible.

[napulen]

The code has been relicensed as BSDv3 now.

[napulen]

Looking at music21/humdrum/spineParser now.

[napulen]

Comprehensive exams got in the way of this, sorry. Continuing this integration now.

When you say as a separate tool, how would you envision the use of this parser? I was thinking something like this:

1. You parse a .krn file (e.g., music21.converter.parse('hello.krn'))
2. Whenever you detect that the file has a **harm spine in it: 2.1 Run the **harm parser on every token that it finds within that spine 2.2 Use the output of the parser (e.g., root, inversion, is_secondary_chord, etc.) to store those properties in a music21 object. For example, at the same level of music21.pitch.Pitch, as a child of music21.note.Note or music21.note.Chord.

This workflow seems to require quite some work, maybe you thought of a simpler way?

Best.

[MarkGotham]

Greetings of the Season to you @napulen , @mscuthbert , and all the merry music21-ers around the world.

Thanks for your great work on this @napulen . I do indeed support this initiative which would fit nicely into music21's ever-improving suite of tools for harmonic analysis (Roman numeral and otherwise). The total coverage now extends to 'formats defined and demonstrated by Dmitri Tymoczko, Trevor De Clercq & David Temperley, and the DCMLab.' https://github.com/cuthbertlab/music21/commit/be057a002ecfa469308049766cc4bdf187763675 It would be great to see **harm added to that list and your code is a much more promising start than the hack I've been using!

Have a look at any of those converters to get a sense of the parts of music21 you need to connect with. Most importantly, and to answer your question, note that there is a roman.RomanNumeral class for storing all the task-specific information: https://web.mit.edu/music21/doc/moduleReference/moduleRoman.html?highlight=roman.romannumeral#music21.roman.RomanNumeral

Note also that music21's handling of the variable conventions for 6th/7th degrees in minor is rapidly improving. Check out music21.roman.Minor67Default: https://web.mit.edu/music21/doc/moduleReference/moduleRoman.html?highlight=roman.romannumeral#music21.roman.Minor67Default

You'll likely want to use that for input, though I don't think that conversion between different conventions is currently supported.

Thanks again for pitching in! As you'll doubtless know, one of music21's strongest assets is the way it can help bring all related work of this kind together. Here's to a bright, and better-coordinated future for the field.

[napulen]

Hi @MarkGotham and @mscuthbert. I had some extra time available this week and decided to go for this integration. Here is the pull request #561.

I had to debug from music21.converter.parse all the way to the stream generation. I leave my notes here for anyone interested in looking at / extending this work:

Extending music21's humdrum parser to support **harm spines

• music21.converter.parse() starts parsing the file. It calls music21.converter.parseFile() when a filename is provided.
• music21.converter.parseFile() figures out the format based on the format parameter or the file extension. Humdrum files with a .krn extension get format="humdrum".
• The file is parsed based on a mapping of the format and a subConverter. The humdrum subconverter is a Humdrum class.
• The humdrum class inherits from HumdrumDataCollection. HumdrumDataCollection has a parse method, which is called within the methods of Humdrum.
• HumdrumDataCollection.parse() extracts every line of the file, that's called a dataStream.
• A dataStream is converted to an eventList, which is an instantiation of every line as a metadata element !!!, global comment !!, or regular line. Blank lines are removed and do not make it to the eventList.
• An eventList is converted into a protoSpine and eventCollections, which are essentially a grid representation of the humdrum data.
• After extracting all protoSpines and eventCollections, these are turned into HumdrumSpine objects. A HumdrumSpine differs from the previous representation in that it handles "splits" of humdrum columns (multiple voices in the same spine that appear throughout the piece and disappear later on).
• A HumdrumSpine is also the "generic" spine class that every spine has at the beginning. Specific spines (**kern or **dynam) get specialized in their own classes (KernSpine or DynamSpine), which have their own parse() method.
• **kern and **dynam spines are processed with their own, overriden, parse() method. For the most part, the parse functions do the same. The crucial difference is the kind of music21 object that they store after parsing.
• **text and **lyric don't have a dedicated class (e.g. TextSpine) because they don't need any special processing of the contents in the parse() function. They get wrapped in generic objects called ElementWrapper. These objects are inserted to the main stream of **kern spines in the appropiate offset. The .content itself of the ElementWrapper becomes the lyric.
• **harm spines will do require a dedicated class, HarmSpine, because they need a special processing of the .content. I made that class with an overriden parse() method. In the parsing process, I intend to turn each **harm annotation into a RomanNumeral object. The HarmParser class is an interface to parse the **harm annotations first, and (easily) transform them into RomanNumeral objects.
• The objects generated through the parse process can not be arbitrary, as the code expects a series of methods and attributes and functions to exist. I believe that any class that inherits from music21.base.Music21Object can satisfy this, although I am not sure.
• Therefore, I found it easier to store **harm annotations as RomanNumeral objects, which already inherit from Harmony (and Harmony from Chord, and Chord from base.Music21Object).
• The conversion between **harm annotations and RomanNumeral strings (which are not always compatible) is done in convertHarmToRomanNumeral(), a function within the HarmSpine class.
• Changes of key are taken into account in HarmSpine.parse as long as they occur within the **harm spine.

[napulen]

Lastly, for parsing **harm annotations, I used my old regex HarmDefs, which you are already familiar with. I added those as a module music21.humdrum.harmparser. I don't mind changing the license of that module according to what suites best for music21.

Given that said, for my own research, I do not use HarmDef anymore. I parse **harm annotations using a newer parser I've made, harmalysis, which is more robust and provides better validation of the syntax.

Specific examples where harmalysis is better than HarmDef:

I64             # handling numeric inversions additionally to the letter inversions used in **harm
IP7             # the quality of an interval is validated (sevenths can't be perfect, only major/minor/diminished/augmented). Interval quality is not validated in HarmDef
Vm9m7           # order of intervals (should be ascending according to **harm) is not validated in HarmDef
c:i             # changes of key within the annotation. This is necessary for parsing **harm-like annotations in MusicXML or MEI, where there is no straightforward mechanism to denote a change of key

... and other things mentioned in the MEC 2020 presentation.

I recommend using harmalysis instead of HarmDef eventually. The drawback is that you need one more dependency on your requirements.txt. I thought you may have reasons for not wanting a new dependency, so I went with HarmDef for this PR.