Add support for Voltas

[hedgert]

I've coded handling of Voltas - however, there is one type that NWC appears to handle but I don't see reference to in MusicXML documentation and Sibelius doesn't seem to handle it. It's a type of ending "D" (default ending according to NWC) - and it's used for a final repeat after DS/DCs have happened - NWC handles the flow for it - and I've coded it into MusicXML but Sibelius ignores a "D" numbered ending - not sure if that's because Sibelius doesn't implement MusicXML properly - or whether NWC have created something that isn't standard - any thoughts? The screenshot I've attached is an NWC example file for voltas and you can see the 2,D ending in bar 6. What NWC does is to use that "Default" ending when it repeats after the DSalCoda back to the Segno - i.e. after the return to the Segno it uses the D labelled ending.

[lasconic]

I never saw a D. indication. I checked Behind Bars by Ellen Gould, regarded as a reference for music engraving, and there is no mention about D, so I guess it's an invention of NWC. Maybe @nwsw can confirm? Since it's in the original, it's ok to put it in the MusicXML but I wouldn't expect a MusicXML importer to do anything useful with it, except for display.

[hedgert]

OK - then I'll leave D in the MusicXML. I attach the nwctxt that is the equivalent of the NWC file above. REPEAT4.txt

[hedgert]

Is this another NWC special? A number in brackets above a repeat close bar - which NWC interprets as going back to the previous section open and repeating the bars in between the number of times in the brackets. I can't find anything in MusicXML for this (although the flow control is there - the visual indication doesn't seem to be, so I wonder if this is non-standard/not MusicXML)?

[lasconic]

The flow control, I gather you are talking about ?http://usermanuals.musicxml.com/MusicXML/MusicXML.htm#CT-MusicXML-repeat.htm

For the text, I can't find anything in Gould but I have seen 3x in many scores. I guess the text could be converted to a simple direction text.

[hedgert]

Thanks - I'll code 3x in place of (3) that NWC exports.

[hedgert]

I'm working on the repeat (3x) and I'm not sure either Sibelius or MuseScore are doing the right thing - before we even get to the repeat - that I think the MusicXML does support. This is my NWC input file: TestLocalRepeatCloseRepeat3.pdf So it has a section bounded by local section open close bars - and the (3) annotation - but ignore the (3) for now. The bar lines bounded bars 2-4 are light-light. Sibelius renders the XML (which I attach below) as (with heavy-heavy barlines): And MuseScore renders it with light-light barlines but no without the double dots for the repeat:

So to my untrained eye - I think they're both wrong - given the XML I've generated. Am I misunderstanding the light-light barline repeats (if so does that mean that NWC's use of them is non-standard)?

Generated XML that I've loaded into both Sibelius and MuseScore is:

LocalRepeatCloseRepeat3.txt

This is the MusicXML reference for the repeat times attribute that I've used http://usermanuals.musicxml.com/MusicXML/Content/EL-MusicXML-repeat.htm

[hedgert]

I think I’m done - but extending my testing to double staves (I’ve testing everything on a single stave so far). I’ve added voltas, fermata/breath marks and the various tempo variances (in NWC those are grouped together for some reason), segnos and I’ve implemented repeating bars (i.e. a pair of double barlines with dots opening and the later closing with a direction to repeat that set of bars n times. Interestingly Sibelius has a bug and doesn’t handle that correctly (you can write it in Sibelius, but even if you export as MusicXML and re-read Sibelius’ own XML back in it doesn’t work) but your MuseScore does handle it correctly. I’ve changed one thing about your implementation in my version (for the repeating bars above): NWC has light-light bars with double dots. Sibelius converts these into heavy-light then light-heavy with dots. MuseScore just interprets them as light double bar lines with no dots. So neither does what NWC does (offering to repeat a block of bars between light-light double dotted barlines). Given that - where your code interprets the various bar types, where NWC codes LocalRepeatOpen and LocalRepeatClose I’ve implemented as heavy-light and light-heavy with a repeat - only for the repeat. Given neither Sibelius nor MuseScore can handle light-light repeats, I don’t believe this takes anything away and it does add support for a block of bars repeated. You can see this in action in my test Nwc2MusicXMLTest_LocalRepeats.java

I’ve added tests for everything I’ve done using JUnit. As you suggested I’ve used a file diff, so if the test fails it doesn’t tell you why - but it does provide assurance as code is changed in the future that these continue to work. And if you want to see what the actual differences are you can use unix diff.

As I say I need to run some tests to ensure that multiple staves work - particularly with voltas. And for that I have a question about the style of your code that I should follow. NWC can produce nonsense music (in nwctxt and musicXML) as it allows you to put different voltas on different staves. NWC even plays the staves independently if that happens - i.e. you see/hear the highlighted note playing in a different bar on different staves at the same time. So my question is: given NWC produces that nonsense, should I just faithfully translate the nonsense into XML (not sure what Sibelius or MuseScore will do with it) or should I reject it - which I could most easily do by only coding the voltas on the first staff across all staves - which would ensure they are all the same. Or something else?

If I’m understanding correctly having done Github commits so far for all my changes, once I’m finished with the above, I push - and then you can see my fork? And I’m assuming you validate what I’ve done in some way before you choose to put the code back into the main branch?

Thanks Tim

On 7 May 2016, at 22:12, Nicolas Froment notifications@github.com wrote:

The flow control, I gather you are talking about ?http://usermanuals.musicxml.com/MusicXML/MusicXML.htm#CT-MusicXML-repeat.htm http://usermanuals.musicxml.com/MusicXML/MusicXML.htm#CT-MusicXML-repeat.htm For the text, I can't find anything in Gould but I have seen 3x in many scores. I guess the text could be converted to a simple direction text.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/lasconic/nwc2musicxml/issues/15#issuecomment-217669165

[lasconic]

Never saw a Light Light repeat. It must be a NWC particularity. So I believe your decision.

Junit, diff, sounds perfect.

Using voltas in the top staff makes sense... too bad if a user wrote some nonsense. We have the same problem a little bit everywhere in this project since it's also possible to put too many notes or not enough in a measure for example...

I see your fork here https://github.com/hedgert/nwc2musicxml but I don't see your commits, so you probably commit only in your local repository. Hopefully on a branch. You can then push your branch on github with git push origin yourbranch (assuming you set the origin and the name of your branch is "branch"). Then you can indeed do a pull request, I will review the code (mainly compile it and run it on a couple of files...) and integrate it in the main repository (mine) in the master branch.

[hedgert]

Thanks Nicholas. I forked and then coded and locally committed - no branch. I thought fork was giving me a separate code base to work on that could be merged later - obviously more to learn about GitHub! But I'll see if I can fix that locally by doing a branch (on another machine where I don't have my local repository) and then checking my changes into the branched version rather than what I've done so far.

And thanks for the guidance on voltas - I'll make the changes to implement that.

Regards Tim

Sent from my iPhone

On 10 May 2016, at 15:42, Nicolas Froment notifications@github.com wrote:

Never saw a Light Light repeat. It must be a NWC particularity. So I believe your decision.

Junit, diff, sounds perfect.

Using voltas in the top staff makes sense... too bad if a user wrote some nonsense. We have the same problem a little bit everywhere in this project since it's also possible to put too many notes or not enough in a measure for example...

I see your fork here https://github.com/hedgert/nwc2musicxml but I don't see your commits, so you probably commit only in your local repository. Hopefully on a branch. You can then push your branch on github with git push origin yourbranch (assuming you set the origin and the name of your branch is "branch"). Then you can indeed do a pull request, I will review the code (mainly compile it and run it on a couple of files...) and integrate it in the main repository (mine) in the master branch.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[hedgert]

I've now created a branch merged my files into it - confirmed the tests all work and then pushed to the branch. Next step is the multi-stave testing and the changes required to repeat voltas from the first staff across all staves.

[hedgert]

Mmm - how far should I take the "apply the top staff voltas" principle? In creating some test files before looking st the code changes, I took my 2 stave test and inserted an extra bar so the voltas are out of line - and then of course the repeat bar lines (eg light-heavy double dotted at the end of the 1st ending Volta were out of line too. Should I cover barline types as well? Effectively that would mean as we go through the first stave storing the bar lines and the voltas - and I guess the segnos too - and then using that stored set instead of what's actually on subsequent staves. Is that taking too far the possibility that someone produces a daft NWC file?

[hedgert]

MuseScore is more picky than Sibelius - but in this instance is it right to be? In testing my Votlas I find all is well the MusicXML is as I expected and Sibelius reads/displays and plays back correctly - but MuseScore rejects the XML as invalid - this is the issue: Support for closing a volta consists of adding an <ending tag to <barline. And where <barline is already defined (as at the end of the bar), then I add it to the existing <barline that NWC2MusicXML was already creating - my addition goes in before what is already there: Sibelius handles it, MuseScore rejects it saying: If I swap the order of the <ending and <bar-style tags round, then MuseScore loads and displays correctly. (Interestingly it looks like MuseScore doesn't handle the variety of possible voltas - a conventional 1-ending, then 2-ending is OK, but my test of 1,2-ending and 3-ending isn't handled properly (1st ending is played then on the 2nd ending it plays through the 1,2 time bar and then straight through the repeat barline and through the 3rd ending) - I appreciate these are MuseScore comments and I'll raise them elsewhere once I've dealt with this. So MuseScore loads and displays this version correctly:

But surely the order of <bar-style and <ending within <barline doesn't matter from a pure XML perspective? So my question is, is it valid MusicXML either way round, (in which case MuseScore is overly picky) or is it correct that the order has to be <bar-style first and then <ending in which case I need to force that order in my code.

[lasconic]

MuseScore is validating the MusicXML with the standard XSD schema... barline is defined with a xs:sequence with bar-style first and ending next http://usermanuals.musicxml.com/MusicXML/MusicXML.htm#CT-MusicXML-barline.htm As far as I understand elements in a sequence must appear in order http://www.w3schools.com/xml/el_sequence.asp and so MuseScore enforces that.

[hedgert]

Thank you - a bit more coding to do then! Can you comment on my previous post - about how far to go in taking direction from the top staff with voltas?

[lasconic]

About top staff voltas, I don't really have a strong opinion. If there are different repeat barlines, volta or coda/segno on different staves, we somehow need to choose. Question is: do we choose at export time or at import time...? Maybe it's not the job of the converter to decide if it's semantically good and it should just try to export as a much information as possible to MusicXML? Or it should take the job, and in this case, yes, take the first (visible?) staff as a reference or make sure that the information is not contradictory between the different staves...

I understand I ask more questions that I give answers :)

[hedgert]

Thanks Nicholas - I don't mind there not being strong opinions I just want to make sure that if there are, I respect them.

Regarding the below it makes me think about my previous proposal - certainly if I do take the top stave I think pointing it out makes sense. But that makes me think - what's the point of forcing the first stage on the other staves - while it may give XML that is technically correct, presumably something has gone wrong with the original music in NWC - and me just aligning voltas, segnos and barlines with the first stave won't fix that, it might even hide it. As now it will be less visually obvious that the original NWC had problems. So maybe the better approach is to convert faithfully and report the issues - so attention is drawn to bad nwctxt before the user gets to their next music app.

You mention tests you'll do before you publish. If there are particular test files you use, would you share them? I can then add JUnit tests for them to make sure before I finish my changes that I haven't broken anything?

Thanks Tim

Sent from my iPhone

On 12 May 2016, at 07:50, Nicolas Froment notifications@github.com wrote:

About top staff voltas, I don't really have a strong opinion. If there are different repeat barlines, volta or coda/segno on different staves, we somehow need to choose. Question is: do we choose at export time or at import time...? Maybe it's not the job of the converter to decide if it's semantically good and it should just try to export as a much information as possible to MusicXML? Or it should take the job, and in this case, yes, take the first (visible?) staff as a reference or make sure that the information is not contradictory between the different staves...

I understand I ask more questions that I give answers :)

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[lasconic]

I don't have many files and most of them are from people sending bug reports, so I don't feel confortable to share them :-/

[hedgert]

I did a pull request and checked all my changes and tests in about a week ago. I fully appreciate you'll need time to review, but not having heard from you, I wanted to check that you're aware/that I haven't done my pull request wrongly!

Sent from my iPhone

On 12 May 2016, at 17:04, Nicolas Froment notifications@github.com wrote:

I don't have many files and most of them are from people sending bug reports, so I don't feel confortable to share them :-/

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[lasconic]

Hi, sorry for the silent week. I will try to take a closer look this week end.

[lasconic]

Done. 5db52dda8173674e8b00d5f26605455ea51bc4a6