GP import: repeats and other directions (da segno al coda, etc.) do not import correctly

[GoogleCodeExporter]

What steps will reproduce the problem?
1. import a Guitar Pro score with repeat markers or other directions such as 
'da capo', 'da segno', and 'coda' sections
2. notes will be missing & not fill the whole song properly

What is the expected output? What do you see instead?
Expect to see correct repetition/flow of sections marked by score directions. 
Instead, directions are ignored and notes are imported directly as if no 
repeats or other directions were given.  Quick & simple test .gp5 file & output 
screenshots attached.

What version of the product are you using? On what operating system?
EOF 1.8RC4, Mac OSX 10.6.8 (built from source, svn 1088)
imported GP5 (5.10) file (exported from GP6)

Please provide any additional information below.
I see some debug notes related to coda/segno/etc. in gp_import.c (line 498-) 
but don't really know C well enough to figure out how to enable them... :( Is 
there a way to display these? I tried './eof -debug' but that seemed to have no 
effect on the output in eof_log.txt

Original issue reported on code.google.com by ea.th1rtyf0ur on 7 Feb 2013 at 11:57

Attachments:

• eof_test.gp5
• eof_gp_import.png

[GoogleCodeExporter]

This has been a stated limitation for a while, but I've actually been working 
on the repeat logic for the past few days.  After that's done, I should be able 
to tweak it for the directional symbols support.

Original comment by raynebc on 8 Feb 2013 at 7:43

• Changed state: Started

[GoogleCodeExporter]

r1089 added support for repeats, which just leaves implementing support for the 
direction symbols.

Original comment by raynebc on 10 Feb 2013 at 7:26

[GoogleCodeExporter]

Any ETA on getting the direction symbols working too? The (ab-) use of repeat 
signs and the direction symbols in GP almost completely satisfies the 
"Pattern/instancing" I had in mind for issue 256.

Original comment by quarns...@gmail.com on 28 Apr 2013 at 11:45

[GoogleCodeExporter]

Oops, mean issue 265

Original comment by quarns...@gmail.com on 28 Apr 2013 at 11:46

[GoogleCodeExporter]

I'll see what I can come up with.

Original comment by raynebc on 28 Apr 2013 at 6:47

[GoogleCodeExporter]

I managed to build EOF locally, but have an issue getting music playback 
working (see comment issue 261). If I can get the music playback working, I can 
look into getting the direction symbols working for this issue as well as 
looking into importing separate GP tracks as different difficulties for issue 
266 myself if you'd like to look at other issues. Please let me know if you're 
ok with that or you'd rather deal with these issues yourself.

Cheers

Original comment by quarns...@gmail.com on 29 Apr 2013 at 10:50

[GoogleCodeExporter]

I would be able to handle the changes for directional symbols and importing 
into just the active track difficulty, but I have no Mac programming 
experience, so I wouldn't be able to help you get it built in that environment.

Original comment by raynebc on 29 Apr 2013 at 2:40

[GoogleCodeExporter]

I didn't mean to imply that you're unable to handle them, just that I'm 
impatient enough to be willing to dive in and try and implement these myself in 
case there are other issues you want to tackle first. :)

BTW, got it up and fully working on my Mac now, so I'm ready to dive in unless 
you think you'll have these implemented in the next couple of days anyway.

Original comment by quarns...@gmail.com on 29 Apr 2013 at 4:05

[GoogleCodeExporter]

I was going to start working on it, so that should be fine.  I did have a 
question though, in the following series of musical directions:

Segno...da capo...da coda...D.S. al coda...capo...coda

It would play from Segno to DS. al coda, from Segno to da coda, and then from 
coda onward, right?  I just want to check if the "da capo" is supposed to be 
ignored unless an "...al capo" direction was reached.

Original comment by raynebc on 29 Apr 2013 at 6:54

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

"Da capo" means "from the top (beginning)", so you can never place a "capo" 
token. But let's replace capo in your example with "segno segno":

Segno...da segno segno...da coda...D.S. al coda...segno segno...coda...(end)

In case it'll will play

Segno -> D.S al coda
Segno -> da coda
coda -> (end)

"al coda" just means "play until you reach a coda sign".

I'm unsure of the operator precedence though. If we switch places of the 
"segno" and "segno segno" pairs so we have:

Segno segno, da segno, da coda, d.s al coda, segno, coda

It'll play:

Segno segno, da segno, segno, coda

If I have this sequence:

Double coda, Segno Segno, Segno + da coda, da segno segno, da capo,  da double 
coda, da  segno, coda

It plays:
Double coda -> da capo
Double coda -> da segno
Segno + da coda, and then jump straight to coda. The da double coda is never 
executed.

Original comment by quarns...@gmail.com on 29 Apr 2013 at 7:48

[GoogleCodeExporter]

If you don't have GP yourself, I can put together a couple of paired test files 
where one has the directions and the other is "flattened" to play back the same 
thin without the directives. This way you have pairs of an input file and an 
expected output file that could be used in a unit testing suite. It's too late 
for me today, but let me know if you'd like me to put these files together and 
I'll do it tomorrow.

Original comment by quarns...@gmail.com on 29 Apr 2013 at 8:27

[GoogleCodeExporter]

I have Guitar Pro, but if you can put together some test files, it would be 
helpful.

Original comment by raynebc on 29 Apr 2013 at 9:07

[GoogleCodeExporter]

Here are a few test files to get you started. In case you want to create more a 
tip is to export to midi, import the midi and export that for creating the 
flattened version.

Original comment by quarns...@gmail.com on 30 Apr 2013 at 7:08

Attachments:

• testfiles.zip

[GoogleCodeExporter]

There's a bug in the current section repeat code also. See the attached test 
file where the first beat in the second measure is not repeated with the BPM 
setting currently specified in the notes.eof file (never mind that the BPM 
itself is wrong at the moment). 

Original comment by quarns...@gmail.com on 30 Apr 2013 at 9:51

Attachments:

• test.zip

[GoogleCodeExporter]

The issue you mentioned in post #14 wasn't a bug so much as a rounding issue.  
With that particular tempo, the note position rounded down so that it was 1ms 
before measure number 2.  I've made a small change to make imported note 
positions round up to the nearest millisecond, and the note properly repeats so 
that it appears 3 times once import is completed.

Thank you for the test files, I'll take a look at them.

Original comment by raynebc on 30 Apr 2013 at 3:53

[GoogleCodeExporter]

I've gotten the initial logic programmed and am beginning testing.  The 
"a_flat" guitar pro file is incorrect though.  Playback in guitar pro confirms 
that the sequential order is 123234512345612345673456789, which luckily is how 
EOF is interpreting it.  All "da SYMBOL" and "da SYMBOL al SYMBOL" markers are 
all single-use, right?  I imagine otherwise it would be impossible to avoid 
infinitely-looping compositions.

Original comment by raynebc on 30 Apr 2013 at 7:32

• Changed state: In_Progress

[GoogleCodeExporter]

The "b_flat" file also didn't match Guitar Pro's playback of "b", so until I 
hear back, I'm just going to compare the playback order with what EOF is 
importing.

Original comment by raynebc on 30 Apr 2013 at 7:41

[GoogleCodeExporter]

Hmm. It's playing back 12345123456732349 for me (GP 6.1.4 r11201 (06/12/2012) 
on OSX). See attached screen recording (no audio capture apparently).

Don't know whether from a musical theory perspective your version or mine is 
the correct one.

Original comment by quarns...@gmail.com on 30 Apr 2013 at 7:48

Attachments:

• [a - Bredband.m4v](https://storage.googleapis.com/google-code-attachments/editor-on-fire/issue-260/comment-18/a - Bredband.m4v)

[GoogleCodeExporter]

And yes, all "da SYMBOL" markers are single use.

Original comment by quarns...@gmail.com on 30 Apr 2013 at 7:51

[GoogleCodeExporter]

I'm using Guitar Pro 5.2.  I've triple checked the order in which it plays the 
measures in sample file a.  I have to wonder if the version of Guitar Pro you 
have doesn't play it correctly, perhaps it's a bug or a setting.  In the video 
you posted:

1.  After playing measure 3 for the first time, it didn't jump to Segno Segno.
2.  After playing measure 6 for the first time, it didn't jump to Double Coda.
3.  After playing measure 7 for the first time, it jumped to Segno and THEN 
decided to honor the jump to Segno Segno after playing measure 3.  It then 
honored the jump to Coda, even though there was no "da ... al Coda" symbol 
placed in the composition.

I'm having a hard time understanding why the version of Guitar Pro you have 
seems to not follow any consistent rules.

Also, in sample file c, Guitar Pro 5.2 is ending playback after the "fine" 
symbol is reached, even though there is no "da ... al fine" symbol in the 
composition.  My understanding is that the "fine" symbol only ends the 
composition if "da ... al fine" is transcribed.  I have to wonder if Guitar Pro 
5 is also buggy in this respect.  TuxGuitar doesn't handle the navigation 
symbols at all, so I can't check how it would have handled these files.

Original comment by raynebc on 30 Apr 2013 at 8:18

[GoogleCodeExporter]

Coda's should only be executed on the LAST play through according to:

http://en.wikipedia.org/wiki/Coda_(music)
"The instruction "To Coda" indicates that, upon reaching that point during the 
final repetition, the performer is to jump immediately to the separate section 
headed with the coda symbol. For example, this can be used to provide a special 
ending for the final verse of a song."

http://www.myriad-online.com/resources/docs/manual/english/breakjump.htm
"Da Coda (Da ) 
Jump to the bar containing the  coda symbol . However, this jump is performed 
only if :

Either it is the last time this bar is played
or a jump with condition activation (D.C. al Coda or D.S. al Coda ) has been 
performed earlier"

Original comment by quarns...@gmail.com on 30 Apr 2013 at 8:40

[GoogleCodeExporter]

I haven't seen any other source saying that it has to be used only in that way, 
especially since that usage can be explicitly composed using "Da segno al coda" 
or an equivalent symbol.  That also doesn't explain some of the other 
discrepancies I mentioned.  The myriad-online website you linked to also agrees 
that "Da Segno" symbols (this would include Da Segno Segno) are not 
conditional, so in example #1 from my previous post, it should have honored the 
jump to Segno Segno.

I think we're going to have to allow EOF's handling of this stuff to be peer 
reviewed on the Smithy Anvil forum, since Guitar Pro has apparently (based on 
their support forum) had a history of bugs with handling repeats and 
navigational symbols so we can't rely on how it handles things.  If the "Da 
Coda" and "Fine" symbols really have to have special handling to automatically 
activate the last time their respective measures are reached, it may complicate 
the logic significantly.

Original comment by raynebc on 30 Apr 2013 at 9:00

[GoogleCodeExporter]

> especially since that usage can be explicitly composed using "Da segno al 
coda" or an equivalent symbol

Yes, that's the other condition it'll be executed upon.

There's also the reply from a GP developer here explaining 
http://getsatisfaction.guitar-pro.com/arobas_music/topics/gp6_needs_to_fix_a_pro
blem_with_basic_musical_direction?page=1#reply_8447352
"The first coda is "activated" if it's the last time it's is encoutered in the 
score or if there was indication that explicitly activate the first Coda (DC 
alCoda, DS al Coda) "

Original comment by quarns...@gmail.com on 30 Apr 2013 at 9:26

[GoogleCodeExporter]

I suppose that's possible, but that does mean that GP6's possible mis-handling 
of Da Segno from your video capture is still not explained.  After the next 
hotfix is posted, we'll want to ask the other users to critique it and point 
out where EOF and Guitar Pro are handling some of the symbols incorrectly.

Original comment by raynebc on 30 Apr 2013 at 9:36

[GoogleCodeExporter]

These two sheet reading videos also don't execute the "to coda" jump until 
"it's time":

https://www.youtube.com/watch?v=X-CkYROcOpg&t=197
https://www.youtube.com/watch?v=zNaMvJtvYW8&t=448

But besides what the correct way from a music theory perspective is, in this 
case we're specifically trying to import from guitar pro in which case I think 
EOF should try to be GP compatible. As I've only got GP6 myself, that's the 
only one I care about personally, but it might be of value to let the user 
choose GP5 or GP6 compatibility in the preferences if it can't be autodetected 
from the file somehow.

Original comment by quarns...@gmail.com on 30 Apr 2013 at 9:50

[GoogleCodeExporter]

r1139 adds the initial navigational symbol logic.  For now, I'll prefer 
discussion about changes to the logic, whether some symbols only activate in 
certain conditions, automatically in certain conditions, etc. to be done on the 
forum, where there are more music experts that could weigh in on the topic.

I would also like to work on adding support for alternate endings while I'm 
working on this.  I'll work on that soon, but if anybody can whip up some test 
files I'll be able to debug the new logic faster.

Original comment by raynebc on 1 May 2013 at 1:59

[GoogleCodeExporter]

Re: #25, if GP isn't handling the notation correctly, and if different versions 
of GP behave differently, I think it's more appropriate to handle the notation 
as it's intended, and not how GP handles if (if GP's handling is incorrect). 
I've found several GP tabs online that were obviously entered from score books 
(which often include 3 or more coda sections, which GP doesn't currently 
support), and which don't actually play properly in GP (whether because they're 
not entered correctly or GP isn't playing them correctly I'd have to double 
check), and I don't think it's the best use of time/energy to try to mimic the 
playback behavior of improperly notated files. :)

Since I've had many years of music lessons (piano, band, choral, etc.) & am 
quite familiar w/ the terms (and since I raised the issue :D ), I can weight in 
on how they should work. Is there a topic on the forums already started for 
this?

Original comment by ea.th1rtyf0ur on 1 May 2013 at 6:08

[GoogleCodeExporter]

Not as a separate topic (feel free to create a new one) but it's mentioned by 
raynebc here: http://forums.smithyanvil.com/viewtopic.php?pid=27214#p27214

FWIW, all my real songs using musical directions appear to behave as I'd like 
them to both in GP and in EOF now.

I'll whip together a couple of test files for the alternative endings.

Original comment by quarns...@gmail.com on 1 May 2013 at 6:13

[GoogleCodeExporter]

Testfiles for alternative endings attached. Unsure if "f" is behaving as it 
should, otherwise I think they are correct.

Original comment by quarns...@gmail.com on 1 May 2013 at 6:31

Attachments:

• alt_testfiles.zip

[GoogleCodeExporter]

Also of note, the topic mentioned in #23, as well as the last example in the 
2nd video in #25, seem to have a few "examples" that show confusing and/or 
incorrect notation. I'd like to hope that music notation software should mark 
any such glaring errors, but I suppose not all do, so- just a heads up to be 
wary even of "examples" online.

On a slightly different note, here's an example (pardon the poor image) quality 
of a score book w/ multiple codas- not sure how you're coding in the 
coda/double coda/etc. logic, but since it's possible future versions of Guitar 
Pro will support arbitrary numbers of codas/segnos like this, it might be a 
good idea to program in the flexibility to do so now rather than having to 
change it later. Apologies if you've already done it that way. :)

I'll go check out that forum topic.

Original comment by ea.th1rtyf0ur on 1 May 2013 at 6:34

Attachments:

• multi-coda.png

[GoogleCodeExporter]

Re: #27, I agree that we should disregard Guitar Pro's errors and try to have 
the imported notes match proper music theory.  Let me know if the "Da Coda" and 
"Fine" symbols are supposed to automatically take effect if the measure 
containing them is being played for the last time, even if no "Da ... al Coda" 
or "Da ... al Fine" notation had been used.  Implementing handling for this 
usage of these symbols could be complicated, so I don't want to work on it 
unless there's a consensus that's how they're supposed to work.  If you have 
time to post your thoughts on this matter on on the Smithy Anvil forum, it will 
help engage other users in the discussion.

Re: #29, Thanks for the test files.

Re: #30, Guitar Pro 5 (and supposedly 6) doesn't support any navigation symbol 
(ie. Coda) to be defined in more than one position.  My understanding is that's 
why they made "Double Coda" and "Segno Segno" symbols, to give people more 
flexibility around this limitation.  EOF probably won't gain support for 
versions of Guitar Pro higher than 5, since the format is hugely different and 
it would be an awful lot of work when there are easy ways to just convert a GP6 
file to GP5.

Original comment by raynebc on 1 May 2013 at 8:30

• Changed state: Started

[GoogleCodeExporter]

Re: #31/27- while it may be understood that e.g. a 'fine' takes effect after a 
"DC" even w/o an "al Fine", I think it's much better to be explicit with a "DC 
al Fine" if that's what's intended. This seems analogous to "strict" vs "loose" 
modes of HTML coding- browsers can 'handle' unbalanced tags, etc., but they're 
not really correct.

Re: #31/30- yes, there can only be 1 of each symbol, the ones in the example 
I've given are numbered, which makes them distinct. It's a bit hard to read 
(sorry again), but the sections defined are Coda 2 (w/ a DS3 jump at the 
bottom), Coda 3 (w/ a DS4), and Coda 4 (there's also a DS2 jump at the top 
right corner of the image from the first Coda, which is mostly cut off). 
Essentially GP's Double Coda is the same as the 'Coda 2' in this example, but 
by numbering them they're not limited to just 2. So it might be better to use 
some kind of Segno[level]/Coda[level] arrays to keep track of these, instead of 
making separate Segno/Coda and SegnoSegno/DoubleCoda variables, which would be 
more work to expand later on.

Now that I think about it, the above score book example just has "DS3" etc. 
instead of "DS3 al (Coda)3" etc. So I guess the 'al Coda' is merely implied 
instead of explicitly listed for each 'Dal Segno'. For human reading, this is 
probably sufficient, since each DS(x) will necessarily match up to a 
corresponding Da Coda(x), but for software purposes it's not necessarily 
obvious (without doing a lot of pre-parsing) whether there will be a 
corresponding 'Da Coda' or a 'Fine'. And if someone inadvertently enters both 
(w/o GP stopping them from doing so), the result would pretty much be 
"whichever comes first", which I can imagine being a PITA from a programming 
standpoint.

Original comment by ea.th1rtyf0ur on 2 May 2013 at 1:39

[GoogleCodeExporter]

started a thread: http://forums.smithyanvil.com/viewtopic.php?pid=27437#p27437

Original comment by ea.th1rtyf0ur on 2 May 2013 at 4:02

[GoogleCodeExporter]

Regarding alternative endings, I have a sequence like the attached picture in 
guitar pro. The "5x" is not something I'm in control of and GP insists on 
adding it, but what I really want is for it to play it like this:

0 1 0 2 0 3 0 1 (done)

In other words 4 total repetitions with the first alternative being played the 
first and the last time. GP instead plays this as:

0 1 0 2 0 3 0 1 0 (extra 0 here...)

Is my notation correct and GP wrong or is GP right and I need to change 
something? 

Original comment by quarns...@gmail.com on 4 May 2013 at 9:15

Attachments:

• [alternative endings.png](https://storage.googleapis.com/google-code-attachments/editor-on-fire/issue-260/comment-34/alternative endings.png)

[GoogleCodeExporter]

Working it out in my head, I came up with 010203010 being the correct sequence 
as well.  You never defined what the fifth playthrough would be, so it threw in 
the 0 and then a rest measure.  I think what you're wanting would require you 
to make it a 4x repeat instead.

Original comment by raynebc on 4 May 2013 at 6:12

[GoogleCodeExporter]

>  I think what you're wanting would require you to make it a 4x repeat instead.

I would if I could, but GP is in charge of the count and I can't change it when 
alternative endings are used apparently. What "worked" in the end to make it 
only repeat four times was to treat the first measure as an alternative ending 
too, making the 5th repeat that GP forces upon me have no measures at all.

Original comment by quarns...@gmail.com on 4 May 2013 at 6:30

[GoogleCodeExporter]

I couldn't get it to work right in Guitar Pro until I authored it this way.

Original comment by raynebc on 4 May 2013 at 7:29

Attachments:

• gp_alt_endings.jpg

[GoogleCodeExporter]

Re: #34, the notation in #37 is correct (although ending 4/measure 5 should be 
an F (fret 1) to preserve the same note sequence). In the original post, 
grouping repeat endings 1 & 4 put them both on the left side of the repeat bar, 
which forces another repeat afterward- a 5th ending is then implied after the 
final repeat bar, which is why measure 1 was repeated again. Also, each 
'alternate ending' should have its own right-side repeat bar at the end (as in 
#37), unless it is the last ending. Thus:

||: A | (1.) B :|| (2.) C ||  <- A B A C
||: A | (1.) B :|| (2.) C :|| (3.) D || <- A B A C A D

On another note, it looks like any first-repeats are supposed to be skipped 
after a DC al Fine or similar: 
http://learnmusictheory.net/PDFs/pdffiles/01-01-08-RepeatSigns.pdf 
"Traditionally, musicians skip over any first endings after they go back for a 
D.C. (or D.S.) repeat." Now that I think of it, I think I may remember coming 
across some tabs like this that were entered from score books, which Guitar Pro 
did not play correctly (it played through all repeats/endings after a DS when 
it wasn't supposed to).

Original comment by ea.th1rtyf0ur on 7 May 2013 at 2:15

[GoogleCodeExporter]

Blimey. It's not that complex when everything is explicit, and that's probably 
a good habit to have anyway.

Original comment by quarns...@gmail.com on 7 May 2013 at 6:55

[GoogleCodeExporter]

Any progress with the alt endings?

Original comment by quarns...@gmail.com on 17 May 2013 at 10:37

[GoogleCodeExporter]

I hadn't worked on it lately, but I think I grasp ea.th1rtyf0ur's post enough 
to plan the rest of the logic.

Original comment by raynebc on 17 May 2013 at 5:37

[GoogleCodeExporter]

FWIW the attached diff matches GP5 for the song I'm working on (which is not to 
say that it is correct). 

Original comment by quarns...@gmail.com on 17 Jun 2013 at 1:43

Attachments:

• alt.diff

[GoogleCodeExporter]

That looks like it's definitely a good start on the logic, but that leaves the 
logic to cause the highest numbered alternate ending to be activated again 
(ignoring the lowered numbered endings) in the event that a navigational symbol 
like Da Capo causes it to be reached again.  I'm still not sure about the best 
way to do this, as the program would have to identify which alternate endings 
were related to each other.  Maybe we should require something like the 
multiple possible endings having to be adjacent to each other?

Original comment by raynebc on 17 Jun 2013 at 5:38

[GoogleCodeExporter]

I was thinking something along the lines of having each measure keep a count of 
how many times it's been "unwrapped", and possibly keep a status indicating 
whether it will be allowed to unwrap anymore (this can be set to false if a 
higher numbered alternate ending was reached and will take its place every time 
the measure is subsequently accessed).

Original comment by raynebc on 17 Jun 2013 at 5:46

[GoogleCodeExporter]

I've applied quarnster's logic, with very slight alterations, in r1158.

Original comment by raynebc on 17 Jun 2013 at 5:59

[GoogleCodeExporter]

Since the last post, I've done some substantial rewrites of the unwrap logic 
and as far as I can tell it is working perfectly for repeats, navigational 
symbols, etc.  Are there any other needed changes before this issue is closed?

Original comment by raynebc on 11 Jan 2014 at 9:45

[GoogleCodeExporter]

No follow up, so I'll consider this done.

Original comment by raynebc on 20 Jan 2014 at 7:35

• Changed state: Fixed