How to interpret notes according to a key signature?

[nicolai-rostov]

Hi.

How can I use mmp-to-MusicXML to convert MMP to MusicXML in conformity with a given key signature?

For example, note #58 in a MMP file comes out by default in the MusicXML output as an A₄ with a ♯ accidental. Is there some command-line option or tweak to make your tool interpret it as, say, B₄ with a flat in the key signature (F major)?

I understand I could use an external score editor to transpose the output file manually. But I was wondering if your tool could do it automatically, as I'm trying come up with a convert script to use mmp-to-MusicXML as a bridge from MMP to LilyPond.

Thank you

[syncopika]

Hi Nicolai,

Thank you very much for your interest and great question!

At the moment I don't have that functionality but let me see if I can do something about it ;). I think I have an idea for how to implement that feature - I'll keep you posted if I get something working (hopefully either later today or tomorrow).

Thanks again and best regards!

[syncopika]

Hey @nicolai-rostov, If you get a chance, you can check out the updated script in my branch called add-specified-key-signature-support which contains changes to support the functionality you requested.

Try running python convert-mmp.py <filename> -k f, where the -k flag allows you to specify the key signature (right now the options are C (default), G, D, A, E, B, F, Bb, Eb, Ab, Db, Gb, Cb - but please note that I have it so that the key signature argument is expected to be in lowercase, e.g. Db -> db).

Let me know if what I have is still inadequate and I'll try to address it ASAP. If it's all good for now, I can merge my changes (lmk if you'd like to review first though).

Also feel free to let me know if you have any other suggestions for features!

[nicolai-rostov]

Hello @syncopika,

Thank you so much for having addressed my request so promptly. The new -k option is great, but there seems to be a bug somewhere in commit 936fc94. Using -k f as command-line parameters, note #58 (a black key on the piano) is converted to B♮, not B♭. The key signature comes out right (F major), but the pitch itself is wrong. I think an <alter>-1</alter> element (here) is missing from the MusicXML output file.

Input:

<pattern type="1" name="piano" muted="0" steps="16" pos="0">
<note pan="0" len="48" key="58" vol="100" pos="0"/>
</pattern>

Current output:

<pitch>
  <step>B</step>
  <octave>4</octave>
</pitch>

Expected output:

<pitch>
  <step>B</step>
  <alter>-1</alter>
  <octave>4</octave>
</pitch>

Thanks!

[syncopika]

hi @nicolai-rostov,

No problem, thanks very much for the detailed follow up! That is very interesting. Do you think you could post your mmp file that you tested on so I can take a look?

Also if you don't mind, could you do a git pull on the branch to pull the latest changes? I added a new test file (/testfiles/xmltest2.mmp) that has only 1 Bb note just like in your example. The pattern in my new mmp test file looks pretty much the same as your input:

        <pattern steps="16" muted="0" type="1" name="piano" pos="0" len="192" frozen="0">
          <note pan="0" key="58" vol="100" pos="0" len="48"/>
        </pattern>

If you could run python convert-mmp.py testfiles/xmltest2.mmp -k f and let me know if you're still getting a B natural, that'd be great!

At least for me, after I run that, I get this in MuseScore with the resulting xml file:

Additionally, if I edit the pattern in my file to be exactly as you provided, the result is still a Bb for me.

edit: Perhaps you adjusted the master pitch up 1 semitone in LMMS? I was able to reproduce your output by doing that.

[nicolai-rostov]

edit: Perhaps you adjusted the master pitch up 1 semitone in LMMS? I was able to reproduce your output by doing that.

You're right. That is exactly what was wrong with my sample. Sorry about that! I tested commit 1a9bdcc against your sample, and got the same results as you did. It works! Thanks a lot.

But now... it becomes a bit tricky. All currently supported key signatures seem to be working except for 2: namely, G♭ and C♭. Actually, I guess the current code won't work for any other key signature with more than 5 flats or sharps, like C♯ and F♯, as well as the other six with double flats and sharps.

Let's say a key signature is working if no accidentals occur to pitches that belong to its diatonic major scale. So, D♭ (5 flats) is working:

But G♭ (6 flats) and C♭ (7 flats) are not:

Here commit 1a9bdcc wrongly interprets piano white key #59 for the key signature of G♭ as B♮ instead of C♭. And here:

It also interprets white key #48 for the key signature of C♭ as E♮ instead of F♭.

I've attached a tar.gz file with the currently supported diatonic major scales, and also some chromatic ones, in the MMP format, as well as the corresponding currently generated PNG files, for further testing.

I have other ideas too, but, for the sake of completeness, I think we should probably try to get that right first.

Thank you very much for your time and your code. It already covers the vast majority of music there is!

scales_1a9bdcc.tar.gz

[syncopika]

no worries, and thanks again for your thorough feedback and testing (the test files are much appreciated)! I will take a look at this - at a glance it looks like it should be a minor tweak. apologies for not catching it myself!

and correct, for now I'm (at least for the time being) ignoring the other key signatures that you guessed aren't supported 😆.

[syncopika]

ok @nicolai-rostov, hopefully I've gotten this right now lol (I've also added some new tests to compare against expected xml output). Please feel free to try the latest commit.

Thanks!!

[nicolai-rostov]

Great job! Thank you. G♭ and C♭ seem to be working as expected, and everything looks good as of commit b0b2c91 in the diatonic scales for the other supported keys.

I've converted a couple of classical music MMP files too, in major and minor modes, and I can say I'm surprised how well your tool handles them. Obviously, some notes come out wrong in places where things like applied chords, modulation, and key signature changes appear; but there is nothing we can do about that, as MMP files are just unable to represent this sort of information.

Mention of minor mode brings me to another request, please. How about allowing the -k command-line option to accept minor keys as aliases for major keys, so that users don't have to ask themselves what the relative major of a minor key is? This may help a little bit:

MINOR_TO_MAJOR = {
   "abm":  "cb",
   "ebm":  "gb",
   "bbm":  "db",
   "fm":   "ab",
   "cm":   "eb",
   "gm":   "bb",
   "dm":   "f",
   "am":   "c",
   "em":   "g",
   "bm":   "d",
   "fsm":  "a",
   "csm":  "e",
   "gsm":  "b",
   "dsm":  "fs",
   "asm":  "cs"
}

Excuse my Python. The bottom two, F♯ and C♯, are still missing, though. Is it too complicated to add support for them (6 and 7 sharps), as you already have for G♭ and C♭ (6 and 7 flats)?

Thanks again!

[syncopika]

awesome! I'm very glad my tool seems to be working well in general for someone other than me 😄.

yep, that's a very reasonable request 👍 - I'll have a go at adding the other keys and the minor key aliases (thanks for providing the mapping!).

[syncopika]

support for F♯ and C♯ keys has been added now, as well as being able to pass in minor keys to the -k flag. :D

[nicolai-rostov]

Great! Thank you, @syncopika. I've tested major and minor, both natural and melodic, diatonic as well as chromatic, scales across the 15 supported key signatures, and the results are pretty good. All the major and natural minor scales, as well as most melodic minor ones, are right.

As far as I can tell, there remain just 8 notes to fix as of f7da4d6: namely, the raised 6th and/or 7th degrees of the ascending part of the melodic minor scales for the 5 recently added sharp-minor keys:

f♯ (fsm): E♯ instead of F♮;
c♯ (csm): B♯ instead of C♮;
g♯ (gsm): E♯ instead of F♮, and F𝄪 instead of G♮;
d♯ (dsm): B♯ instead of C♮, and C𝄪 instead of D♮;
a♯ (asm): F𝄪 instead of G♮, and G𝄪 instead of A♮.

For example, C-sharp minor:

I've attached the relevant MMP files below.

Thanks! f7da4d6_melodic_sharp-minor.tar.gz

[syncopika]

hi @nicolai-rostov

I'm not too well-versed in music theory so apologies if this is a silly question but do you think it might be possible for someone to want to specify a natural minor instead of a melodic minor (such that they wouldn't care about or want the raised 6th and/or 7th)?

do we need to make a further distinction between melodic and natural minor key options?

[nicolai-rostov]

[...] do you think it might be possible for someone to want to specify a natural minor instead of a melodic minor (such that they wouldn't care about or want the raised 6th and/or 7th)?

Sure, someone at some point might want to specify a particular variation of the minor scales. However, I don't think this is how we should try to solve this issue. To enforce a particular variation may well be a command-line option, not the default behavior. The default behavior should be to assume (if anything) a melodic minor scale.

do we need to make a further distinction between melodic and natural minor key options?

Nope.

[syncopika]

Hi @nicolai-rostov,

Hope all is well!

I made an attempt at implementing your algorithm (steps 1-5) from https://github.com/syncopika/mmp-to-MusicXML/pull/9#issuecomment-2217473730 on a new branch (key-signature-support-new-strategy) if you'd like to take a look - see the __calculate_note_list method in key_sig_note_finder.py from my changes in https://github.com/syncopika/mmp-to-MusicXML/compare/add-specified-key-signature-support...key-signature-support-new-strategy?expand=1 (sorry about the extra noise with all the new tests 🙏).

I'm not sure I fully understood your idea because as I was trying to implement it, it felt like I was simply redoing my previous strategy (but in a slightly different way).

For example, keeping track of diatonics is definitely one way to go about things but I don't think that's much different from my previous strategy of just doing a modulo to get the note and adjusting to the right enharmonic via lookup table and key signature. Either way I think you'd have to look up the correct note based on key right?

Also, I am still not sure after thinking about it a lot more how we can generalize picking out the right enharmonics without having to hardcode the expected ones for each key signature, since it seems a bit arbitrary anyway (which seemed to be implied in https://github.com/syncopika/mmp-to-MusicXML/pull/9#issuecomment-2217515080 that it wouldn't be necessary?). I totally agree with developing generalized solutions to cover as many cases as possible but I'm starting to feel that there may not be one for this particular problem.

Anyhow, maybe my attempt can be of assistance to you in some way and as it is right now in my branch, I believe your cases with the melodic minor scales in https://github.com/syncopika/mmp-to-MusicXML/issues/7#issuecomment-2213948108 are taken care of.

Thanks again and best regards!

[nicolai-rostov]

Dear syncopika,

You did it! Commit 6cc2d0a gets all the 15 major, and all the 15 melodic minor, scales right. I was particularly pleased to see how smoothly your code now handles sharps, and double sharps, in 6th and 7th degrees of the minor ones:

Good job.

I'm not sure I fully understood your idea because as I was trying to implement it, it felt like I was simply redoing my previous strategy (but in a slightly different way).

You fully understood, and have implemented it exactly as I had imagined it. Thanks! I had started doing basically the same thing last night, but was glad to see you did it first, as Python is not my mother language.

For example, keeping track of diatonics is definitely one way to go about things but I don't think that's much different from my previous strategy of just doing a modulo to get the note and adjusting to the right enharmonic via lookup table and key signature.

Maybe you're right. I don't know. Regardless of how much similar they are, the former approach didn't work for all key signatures. The current one does.

Also, I am still not sure after thinking about it a lot more how we can generalize picking out the right enharmonics without having to hardcode the expected ones for each key signature, since it seems a bit arbitrary anyway (which seemed to be implied in #9 (comment) that it wouldn't be necessary?)

We can drop the enharmonics table. Believe me. The question is, rather, whether we need to.

Your code works. I mean, it works admirably well for diatonic notes. There are, it's true, a couple of secondary issues that show up with some chromatic scales, if you want to get picky about it. But, again, maybe we don't have to address them, you know?

There're other issues, like missing note ties across measures, that are more pressing, in my opinion.

I totally agree with developing generalized solutions to cover as many cases as possible but I'm starting to feel that there may not be one for this particular problem.

It's no longer a problem. A solved problem is not a problem. We could close this issue as successfully completed, if you want to.

Thanks!

[syncopika]

Awesome, thanks very much! I will merge our changes then and close this issue.

There're other issues, like missing note ties across measures, that are more pressing, in my opinion.

Indeed, this is a problem that bothers me a bit as well 😄.