Redesigning ly.music

[wbsoft]

Dear friends, I want to take the lead for redesigning ly.music. This issue is to continue the thread at wbsoft/frescobaldi#492, which I'll close.

ly.music currently has two shortcomings:

• it is not easy to build a tree from scratch
• it is not possible to create LilyPond code from scratch (from a tree).

It was initially designed to create a tree structure from an existing document, which is done in the ly.music.read module.

@crantila emphasizes on the fact that basing ly.music around xml.etree or lxml would make it much faster as not a python object is created for every single tree item, and we would have powerful search possibilities for free.

I am willing to design the ly.music tree such that it adheres to simple principles, maybe even without creating a class for every type of element.

We need the following:

1. build a tree from scratch and have it render a valid LilyPond document from scratch
2. build a tree from tokenized source (from a ly.document.Document) and have all tree items know the originating tokens (and thus source position, as every token has the position in the source file in the pos attribute)
3. make changes to a tree by changing attributes or adding/removing child nodes
4. deep copy (parts of) a tree
5. when making changes to a tree that was build from a tokenized source, precisely know what changes need to be done to the originating document.
6. fast iteration over the tree and understanding the music, time etc like it is currently. This is currently handled inside the Item class and subclasses, but it should move outside the tree node classes.
7. very quick find the node that is at a certain source position.
8. (if possible) for a tree that is created from tokenized source: update the tree from the position where source text changes without recreating the whole tree.

Functions like transpose or translate currently iterate over tokens, but they should operate on the tree in the future, and simply change the tree.

When the tree was built from a tokenized source (i.e. the originating tokens are attached to the tree nodes, while their current value has changed) a dedicated module should be able to find out which changes need to be done to the originating document to have it reflect whatever is changed in the tree. (We won't dump a new LilyPond document because that would bluntly overwrite things that a user might have done different, like whitespace or comments that may or may not be present in the tree, and also that would destroy markings or cursor positions for point-and-click when a document is overwritten inside an editor such as Frescobaldi.)

[Levitanus]

I've started to make a simple translator from REAPER selected items, formatted in Reaper's notation view, and discovered the same issue as described here. But then I've found the python folder in the original LilyPond repository. It seems it has the same structural classes as ly.music.items but better implemented and with possibility to render a tree.

Can the team consider using the original LilyPond repo implementation inside this library? I can try to integrate it when have a spare week

[wbsoft]

I think python-ly is essentially deprecated. The quick-ly project is the successor, with a two-way connection between source text and tree-structured musical representation. However, for creating LilyPond source from a music structure LilyPond's own musicexp.py can be used perfectly.

[Levitanus]

@wbsoft , quickly is the tool I need, thank you so much!)))

[wbsoft]

quickly is not yet complete, it is rather in its infancy and very much in flux, but I have some time to devote to it, and eventually it will become the new engine for Frescobaldi's understanding of LilyPond text input.

[Levitanus]

Thanks, maybe I'll find time to help with it. As I much more prefer to have a package-like tool on PyPi that can be mentioned in requirements than a single-folder from the big LilyPond repository)))