rism-digital / verovio

🎵 Music notation engraving library for MEI with MusicXML and Humdrum support and various toolkits (JavaScript, Python)
https://www.verovio.org
GNU Lesser General Public License v3.0
656 stars 180 forks source link

MIDI for listening/data applications #1928

Open craigsapp opened 3 years ago

craigsapp commented 3 years ago

It would be useful to have two MIDI output types from verovio: (1) MIDI for listening purposes, and (2) MIDI for data processing, such as computer analysis.

The listening form should be the default and labeled midi in the output options (--to midi). The data form should be labeled midi-data in the output options (--to midi-data). Currently there is one MIDI output format for verovio, which is basically equivalent to the new midi-data output format.

midi output should have realization of articulations (staccato, accents – including sforzandi) and ornaments as well as dynamics. Ornaments need to be context-dependent on the tempo, for example an accacciatura could be played x milliseconds before the note which follows it (maybe making x an input option to verovio, such as --ornament-duration-ms). Appoggiatura grace notes would be played at the time of the following notes, and the following note's start time would be delayed by the @dur of the preceding appoggiatura grace note. Similarly for mordents (which will be the easiest ornament to implement, the first two notes could have a duration of x milliseconds, with the last one held for the rest of the duration of the non-ornamented note). Long mordents would behave similarly but the first four notes would be short, and the 5th note would fill out the rest of the original notes duration. Trills would be more complicated to implement, but a basic method would be to use the same x millisecond not between notes in the trill, adjusted so that the last note ends on the original note's pitch. Grace notes before or after the trill note would alter the starting and ending notes of the trill (a tied/slurred note before the trill note means to start the trill on the primary note and not the auxiliary note). Turns would place the ornamental notes immediately at the start of the note if placed above the notehead, or delayed to the end of the note if placed between notes.

midi-data would not realize ornaments or articulations that modify the duration of the note, such as staccatos, but this form of the MIDI output could implement dynamics an accent-based articulations. In addition, MIDI Plus encoding of @accid.ges and slurring information could be used to preserve that information which is otherwise lost in a MIDI rendering of notation. See Beyond MIDI: A Handbook of Musical Codes, chapter 5 for the specification (which I will summarize in this issue for developing the algorithms).


Summary of things that should be implemented for midi and midi-data renderings. The following checklists can be filled in as such features are implemented (by replacing the space with an x inside of the square brackets in the Markdown text for the post).

Performance features to render in midi only:

Performance features to include in both midi and midi-data outputs:

Meta messages to add to MIDI (if not done already). This is most useful for midi-data, but should also be added to midi output as well.

Other considerations:

craigsapp commented 3 years ago

MidiPlus implementation notes:

The midifile library that is used in the MIDI rendering process has two functions MidiMessage::setSpelling() and MidiMessage::getSpelling() that utilize MidiPlus to encode the accidentals for a note within the bottom two bits of the velocity parameter:

https://github.com/craigsapp/midifile/blob/97405c8d4dfbded33d81e5108a5e8c1abf29df93/include/MidiMessage.h#L68-L69

https://github.com/craigsapp/midifile/blob/97405c8d4dfbded33d81e5108a5e8c1abf29df93/src/MidiMessage.cpp#L1226-L1484

This is an implementation of Table 5.3 on page 102 of Beyond MIDI:

Screen Shot 2021-01-18 at 8 43 22 AM

This table shows examples for the fourth octave starting at MIDI note 60. If MIDI note 60 has the bottom two bits of the note attack velocity set to 10, then the note should be interpreted as C4. For D-double-flat, the bits are 01 and for B-sharp they are 11. This system allows for encoding of all accidents through double sharp/flats plus F-triple-sharp.

In the midifile code, an example usage would be to set the note@pname and note@accid (or technically note@accid.ges) parameters from MEI data:

    MidiMessage message;
    message.setKeyNumber(60);
    int pname = 0;   // C=0, D=1, E=2, F=3, G=4, A=5, B=6
    int accid = 0;      // 0=natural, 1=sharp, -1=flat, 2=double-sharp, -2=double-flat

    // set note to C4:
    message.setSpelling(pname, accid);

    // set to D-double-flat:
    message.setSpelling(1, -2);

    // set to B-sharp:
    message.setSpelling(6, 1);

Not relevant for implementation of the midi-data rendering, here is how to extract the spelling:

    message.getSpelling(pname, accid);

This will fill in pname with the diatonic pitch class and accid with the chromatic alteration.

craigsapp commented 3 years ago

See issue https://github.com/music-encoding/music-encoding/issues/764, since @accidupper.ges and @accidlower.ges need to be implemented for this issue.

craigsapp commented 4 days ago

Also note ornament example sampler in issue #3754