thSoft
commented
6 years ago Wow, I was not aware of multi-level includes working this way. According to the documentation (emphasis mine):
Files which are to be included can also contain \include statements of their own. By default, these second-level \include statements are not interpreted until they have been brought into the main file, so the file names they specify must all be relative to the directory containing the main file, not the directory containing the included file. However, this behavior can be changed globally by passing the option ‘-drelative-includes’ option at the command line (or by adding #(ly:set-option 'relative-includes #t) at the top of the main input file). When relative-includes is set to #t, the path for each \include command will be taken relative to the file containing that command. This behavior is recommended and it will become the default behavior in a future version of lilypond. Files relative to the main directory and files relative to some other directory may both be \included by setting relative-includes to #t or #f at appropriate places in the files. For example, if a general library, libA, has been created which itself uses sub-files which are \included by the entry file of that library, those \include statements will need to be preceded by #(ly:set-option #relative-includes #t) so they are interpreted correctly when brought into the main .ly file. [...]
It seems irrealistically complex for Elysium to support changing relative-includes in the middle of a file (!). I think we should invoke LilyPond with -drelative-includes which LilyPond itself recommends and issue a warning for setting relative-includes anywhere in LilyPond files to any value. The question is: is there anyone at all who relies on relative-includes being false by default? Many files in Mutopia set it explicitly to true.
I think the Elysium linking problem is far from solved (and basically unsolvable). I attach an illustrating example. example.zip The problem is that LilyPond seems to process all includes relative to the main file. We have A including B in the main folder and A including B in a subfolder. The main file in the main file includes A in the subfolder, but A's include interpreted relative to the main file is B in the main folder rather than in the subfolder.
And this example does not even have includes pointing to outside the workspace or similar complications.
What are the implications? The current implementation of linking and scoping is plain wrong. Maybe it does not matter that much because multi-step includes are not used that much. So a regular user may not notice it at all. The logic for determining files to recompile will not be perfect. (A change in B in the main folder should mark the main file as dirty...) This may be not too problematic either, because the user generaly knows which files belong together and will recompile the main file. I am most concerned about the refactoring operations, because they try to do something which may be impossible. A consistent refactoring may not be possible at all. Assume, we detect that A in the subfolder really includes B in the main folder, so renaming B would have to change the include in A in the subfolder. But the same file could be included from somewhere else where there is a B which is not renamed...
Anyway, navigation in Elysium will not work! Everything that is included (relatively from a different folder) and is not on the default path or search path will be wrong, because we cannot know from where it is used.