\trill & some other articulations are not recognized

[GoogleCodeExporter]

Put \trill anywhere into your score, this produces the error message "Unknown 
variable"

What version of the product are you using? On what operating system?
Elysium 0.4.0.201404280105
Lilypond 2.18.2
Windows 7

Original issue reported on code.google.com by sirhenry3591 on 19 Jul 2014 at 7:29

[GoogleCodeExporter]

Problem is caused by unusual variable declaration in script-ini.ly:

tenuto = #(make-articulation "tenuto")
thumb = \finger \markup \scale #(cons (magstep 5) (magstep 5))
                        \musicglyph #"scripts.thumb"
trill = #(make-articulation "trill")
turn = #(make-articulation "turn")
upbow = #(make-articulation "upbow")
upmordent = #(make-articulation "upmordent")
upprall = #(make-articulation "upprall")
varcoda = #(make-articulation "varcoda")
verylongfermata = #(make-articulation "verylongfermata")

All definitions after 'thumb' are "eaten up" by the parser. Proved by 
commenting out the thumb lines. Grammar needs to be fixed.

Original comment by sirhenry3591 on 26 Jul 2014 at 7:50

[GoogleCodeExporter]

Wow, thanks for the investigation! The \markup grammar rules are known to be 
very hacky (starting from LilyPond.xtext:50), so it would be great if you could 
help clean them up!

Original comment by harmathdenes on 26 Jul 2014 at 9:23

[GoogleCodeExporter]

Original comment by harmathdenes on 14 Nov 2014 at 10:15

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

As a local workaround, move the thumb definition to the end of the file. I 
added a comment regarding the same problem to issue 55.

Original comment by nit...@itemis.de on 15 Mar 2015 at 12:18

[GoogleCodeExporter]

Thanks nittka but it won't help us much. Users upgrade lilypond and then
the change is lost.

I have been trying hard during the last months but I have not come to a
workable solution so I am about to give up. There is no way until Xtext
gets redone on the basis of Antlr 4. See
https://www.eclipse.org/forums/index.php/t/640630/. Anybody got any news
about this?

Cheers, Wernke

2015-03-15 13:18 GMT+01:00 <elysium.eclipselabs.org@codespot.com>:

Original comment by sirhenry3591 on 16 Mar 2015 at 2:36

[nittka]

Can you elaborate on the \markup grammar rule being hacky? I would not like to start from scratch when approaching the problem. Is there any other documentation on the original grammar other than http://lilypond.org/doc/v2.18/Documentation/contributor/lilypond-grammar? Basically, I am jumping into an involved grammar without knowing the language at all.

For this particular issue distinguishing between Assignment and PropertyAssignment or more importantly recognising the end of a markup seem the key. Given Xtext's general workflow of lexing, parsing, indexing, linking, there are several potential starting points.

One dirty hack I could imagine is splitting up the white space rule into white spaces that end with new lines and white spaces that do not end with new lines (i.e. indentation). Hiding them for different rules in the grammar might allow the backtracking to find the correct alternative. This idea is based on the assumption that assignements (that we want to cross reference) are really made on root level and hence the name has no indentation. Or to look at it from the other direction, a property assignment as part of a makro will be properly indented. This would be a syntactic indicator that could be used without changing the grammar completely or implementing the parser completely by hand.

However, splitting up the white space rule is not without risk - especially when it comes to formatting etc.

A completely different approach would be to leave the parser as it is, but adapt the creation of the index (resource description). In case we stumble upon a property assignment, we analyse the object and check if it may not have been a normal assignemt after all and put it into the index.

I guess the best solution would be having a grammar that yields a model where all elements have the correct type, but my feeling tells me that this is way out of scope. Given the above mentioned workflow it may even be next to impossible because it would be necessary to link the model while instantiating the object tree (in order to determine the number of arguments expected, otherwise you might not know if something is a function argument...).

[thSoft]

Yes, the main problem would be to detect how many arguments a Scheme function has. @wzurborg has made some investigations, but Scheme is very complex. Wernke, could you please describe your findings? Meanwhile I'm trying to document the grammar a bit...

[thSoft]

Some links that may be relevant:

• http://www.lilypond.org/doc/v2.19/Documentation/extending/markup-functions
• http://www.lilypond.org/doc/v2.19/Documentation/extending/lilypond-scheme-syntax
• http://www.scheme.com/tspl2d/grammar.html

[nittka]

I have not worked on the full grammar, but first tried to adapt the existing one to work for more cases which worked out pretty well. Please try out my grammar changes to be found at

https://github.com/nittka/elysium/blob/markupHack/org.elysium/src/org/elysium/LilyPond.xtext

I added some brief comments in the grammar as well.

[thSoft]

Wow, the result of the integration test looks awesome with your modifications! Please open a PR for discussing them. In the meantime, I ignored the src-gen folders, I hope collaborating on the grammar will be easier this way.

[nittka]

Is it correct that the integration test files in snippets come from http://lsr.di.unimi.it/LSR/Search? I just downloaded the current database and it contains more than 800 files now. Also, the number of failing tests is much higher here. Maybe it is time to update the test cases as well?

[thSoft]

Not exactly, they are from the official LilyPond source tree of the latest stable release (2.18.2 as of now). I updated them in 9c5248d.

[thSoft]

Unfortunately, #78 did not fix this issue, \trill, \upbow etc. are still unrecognized.

[nittka]

Too bad. I did not realize this because I used the local workaround described above - moving the \thumb-definition to the end of the file... I remember saying something about more elaborate regression tests ;-)

When playing around a little more last night, I also saw that single line comments for scheme are actually starting with a single semicolon rather than two. This will be quite a challenge because a single semicolon may well appear in regular LilyPond code (e.g. lyrics).

[nittka]

I replaced my local script-init.ly with the version from the test project. On my machine \trill and \upbow are indeed recognized! I guess antlr compiles different parsers on our machines, possibly due to backtracking and timeout settings. Could you try increasing the timeouts in the mwe-file? If this does not help, we should compare the semantic models calculated from the file.

I still think that not checking in the generated code is the right way to go...

[thSoft]

Sorry, I regenerated the language infrastructure once again and now \trill etc. work for me too. (I'm sure it was generated last time also, so it might have been some Eclipse caching issue?)

[thSoft]

Fixd by #78 for now. Full semantic handling of defining markup commands is a larger task, maybe we should reopen #29 when it seems feasible.