How to highlight a portion of the score?

[hugbug]

In my app (which controls Yamaha's digital pianos) the user loads a midi-file for playback and a corresponding musicxml-file is loaded into Lomse score object. Then user can start playback on piano, the program receives playback position from piano via midi-events and moves current playback position in the score using interactor->move_tempo_line_and_scroll_if_necessary - this works perfectly.

Now there is a feature where user can select a portion of the score for practice. When it hits playback button the piano plays the selected portion over and over again (it actually waits for player to push correct keys). The program continues correctly displaying position with the above Lomse function. All good here.

In the original Yamaha application there are two orange vertical lines - one at the beginning and one at the end of the selection. The beat which is currently being played is painted with the orange background.

Screenshot from Yamaha app:

I wonder what are my options to highlight selected portion of the score in Lomse?

• Is it possible to draw vertical lines (of custom color) at certain measure/beat positions? That would be similar to Yamaha app and is very nice looking. When #168 get implemented I can use the same color for both tempo rectangle and selection lines but for now I need to use another color for selection.
• Maybe I can change the color of notes in the selection? That probably wouldn't look as nice as the first alternative but probably can be implemented with current Lomse features. I would need to find out which notes belong to selection. The starting and ending position of the selection is provided as "measure/beat" (as required by piano API).

[cecilios]

Yes, is it possible to draw vertical lines at any point in the score, of any color , line type and line endings, and to remove them when desired. It is just inserting/deleting line objects in the document model at the desired places. The drawback is that this approach requires good knowledge of the document model. Right now I'm busy but in a couple of days I will post some sample code for doing it.

As drawing/removing a pair of markers is a very common need (e.g. to select a portion for playback or other uses) probably it would be useful to add a couple of simple methods for this to the API. But it is necessary to assess what the most common needs may be so that the API be useful and simple. Ideas for this are welcome.

[cecilios]

I have added sample "drawlines" (in examples/samples). It is a simple program to display an score. In the menu there is an option to add colored marker lines to the score, and another option to remove them.

The importante part are methods MainWindow::on_add_lines() and MainWindow::on_remove_lines() in file drawlines.cpp. I wrote the sample very quickly and can be improved. But its in an example of the code you could use for drawing marker lines and removing them.

While coding this sample I've noticed that lomse does not have a suitable API for score manipulation, at least for the most common expected operations. For instance, it is complex to position the cursor on a given measure. I have opened issue #208 to define this API. Any collaboration to define the API is welcome. Examples of use cases that you would like to be considered or ideas to outline the API would be of great help!

[hugbug]

Thank you very much, this is incredibly useful!

In my application the start and end position of selection are defined as measure/beat (via piano midi events). I have to figure out how to move cursor to certain measure/beat. I need to put the first line to the beginning of certain measure/beat and the end line to the end of other measure/beat (if that's possible).

My progress so far:

Orange is the tempo line, blue and green are selection lines. As you can see the selection lines cover only the first instrument. I changed the height of lines from 70 to 180:

That's much better.

Two open questions: 1) How to move cursor to measure/beat (optionally to the end of beat). 2) How to calculate proper height of lines to cover all instruments?

[cecilios]

As to your questions:

How to move cursor to measure/beat (optionally to the end of beat).

There are several options for this:

• Method ScoreCursor::to_measure(int measure, int instr, int staff) allows you to position at start of any measure. From there, you can move forward or backwards to the desired point.

• Method ScoreCursor::to_time(int iInstr, int iStaff, TimeUnits rTargetTime)allows you to move to any time position. But currently, there are no methods to deal with beats, only with time position. You will have to convert beats to time duration by taking into account current time signature.

• Method ScoreCursor::point_to(ImoId nId) can be used to move to end of score by providing a non existing Id such as k_no_imoid.

Leave me a couple of days to document the current cursor API and then you can decide the best options for your app and we could discuss further needs if required.

How to calculate proper height of lines to cover all instruments?

This is a conceptually complex task although its implementation could be simple. This is because you will have to deal with two internal models: the document model and the graphic model.

The document model knows nothing about how the document will be rendered and thus there is no knowledge to determine the height of a line to cover all instruments. At document level you can only deal with local measurements in tenths of staff line spacing and with points referred to an object position. But currently there is no information to link two unrelated points such as top staff of first instrument and last staff of last instrument.

To get that information you will have to query the graphic model, i.e. by accessing the system box and taking its height. Then you will have to convert this box height (in logical units, cents of one millimeter) to tenths (no problem, there are methods for this) and with this information decide the height to assign to the line in the document model.

These type of computations should be restricted to lomse internals, as they can create a lot of backwards incompatibility problems when there are changes in the Lomse internal models. Therefore, I will not support this approach. I would prefer to code an API method for drawing and removing line markers on the score. Let's define an initial specification suitable for your needs and general enough for other uses and I will code it in a few days. If you agree please open an issue for this API method with your initial needs and some generality for other uses. For instance, marker styles (line, open/close brackets, open/close parenthesis, other), marker heights (one staff, all saves for an instrument, all staves from instrument i to instrument j, other). Also it is necessary to consider if these markers have to be just visual effects, overlayed on the score, without representation in the document model or, by the contrary, these markers are real objects in the document model, such as the lines drawn in the 'drawlines' sample.

[hugbug]

Thanks again.

I've found that positioning to measure/beat is easy:

TimeUnits timepos = ScoreAlgorithms::get_timepos_for(score, measure, beat);
cursor.to_time(0, 0, timepos);

There is however an issue with the approach from the example. It is based on add_attachment to an existing object. If however there is no object at certain beat position the cursor.staffobj() returns NULL and we can't add.

Is there another possibility to add a new object to the staff at certain time position?

I would prefer to code an API method for drawing and removing line markers on the score... If you agree please open an issue for this API method.

Great, I'll post an issue.

[cecilios]

Thanks! I forgot about ScoreAlgorithms::get_timepos_for(score, measure, beat); It's a perfectly valid solution.

The solution in the sample is based on add_attachment because lines are auxiliary objects that can not be placed directly on the staff. They need to go attached to an staff object. But when there is no object at certain beat position you can always insert an empty staff object and attach the line to it.

It is not possible to directly insert staff objects at specific time positions. Objects are inserted before or after an existing object or appended at the end of the score. The time position of an inserted object is implied by the existing objects before the insertion point. Consider inserting notes. You can not place a note at a given time pos. You just insert a note after another one, and the resulting time position will be given by the previous note time position plus its duration. A timepos can be occupied by a long duration note. Consider a 4/4 time signature score with one whole note. Beats 2, 3 and 4 are occupied by the note. If you points cursor to the second beat it will move to first object after the whole note and as the time position is greater than the requested one it returns nullptr to signal that that position is invalid.

In previous example, the only posibility to insert an object at second beat is by starting a second voice. You must point to the whole note and insert after it an invisible quarter rest in voice 2, and then the empty staff object, also in voice 2, to attach the line to it.

The complexity of traversing a real score with many parts and inserting objects at specific time positions is big and IMO is not required in real score edition scenarios. Perhaps hig level API method for very common operations could be considered. But working at low level, I would suggest moving to the barline where the measure starts, and from there advance to the desired beat by iteration, until you find an object whose timepos is equal or greater than the desired beat timepos, and attach the line to it. Something as (not tested):

cursor.to_measure(measure, instr, staff);   //will point to first object after the barline
pAt = cursor.getstaffobj();
if (pAt == nullptr)
{
    //nothing after the barline. You are at end of the instrument.
    //Append an empty staffobj to attach later the line to it
    pAt = pInstr->add_spacer(0); 
}
else
{
    //pointing to first object after barline. Locate beat time
    TimeUnits beatTimepos = ScoreAlgorithms::get_timepos_for(score, measure, beat);
    while (!cursor.is_at_end_of_score() && cursor.time() < beatTimepos)
    {
        cursor.move_next();
    }

    //here timepos is equal or greater than desired timepos. Move to right instrument/staff
    while (!cursor.is_at_end_of_score() && cursor.instrument() != instr && cursor.staff() != staff)
    {
        cursor.move_next();
    }

    //here you are at first staff object in the desired instrument/staff with timepos equal or
    //greater than the desired beat time pos. Or you are at end of score 
    if (!cursor.is_at_end_of_score())
    {
        //You are at end of the instrument.
        //Append an empty staffobj to attach later the line to it
        pAt = pInstr->add_spacer(0); 
    }
    else
        pAt = cursor.getstaffobj();

    //now attach the line
    pAt->add_attachment(pDoc, line);

When comparing time positions be aware that they are real numbers. Consider an error epsilon when looking for equality.

[hugbug]

But working at low level, I would suggest moving to the barline where the measure starts, and from there advance to the desired beat by iteration, until you find an object whose timepos is equal or greater than the desired beat timepos, and attach the line to it.

During playback the tempo line can be positioned at any beat, even when there are no notes on any instrument at that beat:

I guess the approach find suitable object to attach to doesn't always work if the want to draw lines at any beat?

the only posibility to insert an object at second beat is by starting a second voice. You must point to the whole note and insert after it an invisible quarter rest in voice 2, and then the empty staff object, also in voice 2, to attach the line to it.

Can you please give an example of this? Or is this the big and complex thing you meant above?

210, if implemented will draw that code obsolete though. Please don't bother with the example if you are going to work on visual markers.

[cecilios]

During playback the tempo line can be positioned at any beat, even when there are no notes on any instrument at that beat

Yes, tempo line is a visual effect and does not use score objects for positioning. It is a picture overlayed on the score at any desired position (logical units). It knows nothing about timepos, mesures, beats or tenths. But ScoreCursor is a kind of iterator over the collection of staff objects and thus only can point to existing objects and not to empty time positions. Instead of returning nullptr when an object does not exist in a given timepos it could have returned the nearest object to that timepos but will never return a non-existing object!

I have started to work on #210 so I would suggets to forget about inserting lines. In a few days, I expect to have a first implementation of the add_visual_market() method to test and experiment with it.

[hugbug]

I have started to work on #210

That's great, thank you!

[hugbug]

Closing this issue then.