First/Second endings in DCML conversion

malcolmsailor commented 2 years ago

music21 version

8.0.0a12

Problem summary

I've noticed that the TSV converter implemented by @MarkGotham and updated by myself doesn't take any account of first and second endings. When there are first and second endings in the DCML files, it simply puts the contents of both endings into the numbered measures. (Whereas, according to my understanding of the RomanText standard, it should label first endings "a" and second endings "b".)

Steps to reproduce

Here's a simple TSV that allows us to reproduce this behavior (it's the 'volta' column that indicates first/second endings):

mc  mn  mc_onset    mn_onset    timesig     staff   voice   volta   label   globalkey   localkey    pedal   chord   special     numeral     form    figbass     changes     relativeroot    cadence     phraseend   chord_type  globalkey_is_minor  localkey_is_minor   chord_tones     added_tones     root    bass_note
 1   1         0           0    2/4                                 .C.I    C           I                   I                   I                                                                       FALSE                                                                                                   
 2   2         0           0    2/4                             1   V       C           I                   V                   V                                                                       FALSE                                                                                                   
 3   2         0           0    2/4                             2   I       C           I                   I                   I                                                                       FALSE

If we put this into a file and run the following commands:

>>> from music21 import romanText
>>> f = "music21/romanText/tsvEg_v2_repeats.tsv"
>>> stream = romanText.tsvConverter.TsvHandler(f, dcml_version=2).toM21Stream()
>>> converter.subConverters.ConverterRomanText().write(stream, 'romanText', fp="temp.txt")

The contents of "temp.txt" are then:

Composer: Composer unknown
Title: Title unknown
Analyst: 
Proofreader: 

Time Signature: 2/4
m1 C: I
m2 V I

Note that measure 2 contains both V (from ending 1) and I (from ending 2) at beat 1.

I am happy to implement a fix myself but I'm wondering if anybody has any pointers to where I can read about the best way of creating/manipulating endings in music21. Thanks!

gregchapman-dev commented 2 years ago

I am happy to implement a fix myself but I'm wondering if anybody has any pointers to where I can read about the best way of creating/manipulating endings in music21. Thanks!

You’re looking for RepeatBracket, in music21/spanner.py. Each RepeatBracket is one ending, and the spanned elements are the Measures in that ending. The docs for RepeatBracket in spanner.py are pretty good.

Greg

On Sep 7, 2022, at 10:41 AM, malcolmsailor @.***> wrote:

music21 version

8.0.0a12

Problem summary

I've noticed that the TSV converter implemented by @MarkGotham https://github.com/MarkGotham and updated by myself doesn't take any account of first and second endings. When there are first and second endings in the DCML files, it simply puts the contents of both endings into the numbered measures. (Whereas, according to my understanding of the RomanText standard, it should label first endings "a" and second endings "b".)

Steps to reproduce

Here's a simple TSV that allows us to reproduce this behavior (it's the 'volta' column that indicates first/second endings):

mc mn mc_onset mn_onset timesig staff voice volta label globalkey localkey pedal chord special numeral form figbass changes relativeroot cadence phraseend chord_type globalkey_is_minor localkey_is_minor chord_tones added_tones root bass_note 1 1 0 0 2/4 .C.I C I I I FALSE
2 2 0 0 2/4 1 V C I V V FALSE
2 2 0 0 2/4 2 I C I I I FALSE

If we put this into a file and run the following commands:

from music21 import romanText f = "music21/romanText/tsvEg_v2_repeats.tsv" stream = romanText.tsvConverter.TsvHandler(f, dcml_version=2).toM21Stream() converter.subConverters.ConverterRomanText().write(stream, 'romanText', fp="temp.txt") The contents of "temp.txt" are then:

Composer: Composer unknown Title: Title unknown Analyst: Proofreader:

Time Signature: 2/4 m1 C: I m2 V I Note that measure 2 contains both V (from ending 1) and I (from ending 2) at beat 1.

I am happy to implement a fix myself but I'm wondering if anybody has any pointers to where I can read about the best way of creating/manipulating endings in music21. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/cuthbertLab/music21/issues/1412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR6X47EHM5KQQTL47RUILSTV5DHUZANCNFSM6AAAAAAQG7NXUA. You are receiving this because you are subscribed to this thread.

malcolmsailor commented 2 years ago

Alright, I implemented repeat brackets here but evidently we should also insert the appropriate repeat bars as well. I've been in touch with Johannes about the best way of inferring these from the DCML files (we can infer that there should be an end repeat at the end of a first ending, but how do we know where the corresponding start repeat should be?). I will submit a PR after that is also complete.

MarkGotham commented 2 years ago

Thanks @malcolmsailor, I hope Johannes has an answer. If not, then perhaps a hacky workaround:

take rntxt and score
compare
transfer all relevant repeat marks from score (presumed to be complete) to analysis
re-write rntxt with these additions

Not perfect, but connects with wider alignment issues that will need support of this kind, as discussed elsewhere. Thanks again, in haste.

johentsch commented 2 years ago

Hi there, repeats are indicated in the TSV files contained in the measures folders that are part of DCML v2.

The next update of ms3 will by default include a quarterbeats in all TSV files and these exclude all endings except the 2nd in the computation of a continuous offset. I don't know how relevant that is to this discussion but I thought I bring it up.

malcolmsailor commented 2 years ago

As Johannes says, it currently stand it requires two files to read the harmonies and the repeat locations from the DCML files. By email, I inquired about the possibility of adding a new column to the DCML TSV files so that we wouldn't need to read two files at once.

But reading two files at once wouldn't be the end of the world. We could do as follows:

add an optional measuresPath kwarg to TSVHandler that takes the path to the measures tsv file.
if measuresPath is not passed, and we come across endings,
- emit a warning
- put all but the last ending (not sure if there are ever more than 2) into an editorial comment or something (it seems to me that if we must omit an ending, it is better to omit the first ending, so that the music will flow smoothly from the section before the endings into the section after the endings).

(We could write the first ending as a romantext formatted text string so that if the file is written out again, we could include it as a "Note:" annotation, e.g., something like

Note: first ending omitted due to missing measures tsv
Note: m5a V
Note: m6a I

Thoughts?

mscuthbert commented 2 years ago

Hi @johentsch -- is the DCML/TSV specification published beyond https://dcmlab.github.io/standards/build/html/reference/specs.html -- this is the first I'm hearing about reading in multiple files in order to get an analysis from DCML/TSV.

I don't think music21's converter should support things not in a spec reference; this comes from experience trying to work with ABC when its spec was rapidly changing. The place for rapid development under changing circumstances would be in a separate tool that can call music21 rather than in the music21 library itself.

Note also that the conversion of TSV in music21 needs to be from TSV to Music21's Stream/Element representation, and from there it can go to RomanText or whatever else, so the discussion of how to express the Note: fields in RomanText output is premature -- it should be about what structure in music21 will express the note field (is it a comment in .editorial? Is it a separate object like HumdrumTandem expressions use; is it a new standard ParsingInformation object?) and then RomanText can be updated to output that object as a Note field.

Thanks!

malcolmsailor commented 2 years ago

I don't think music21's converter should support things not in a spec reference

That seems sensible. Since we do need to do something with 1st/2nd endings (because as it currently stands they are just both written into the same measures), it seems like there are two options:

insert ending brackets for the 1st and 2nd endings (based on the volta column present in the DCML tsv files) but without any associated repeat bars---this is what I already more or less implemented.
put the 1st (or 1st and 2nd, if there is a 3rd ending, etc.) ending into some sort of metadata as I suggested above.

The place for rapid development under changing circumstances would be in a separate tool that can call music21 rather than in the music21 library itself.

+1 that seems right.

Note also that the conversion of TSV in music21 needs to be from TSV to Music21's Stream/Element representation, and from there it can go to RomanText or whatever else, so the discussion of how to express the Note: fields in RomanText output is premature

You're correct, of course!

what structure in music21 will express the note field

Presently, Note: annotations are stored in a RomanTextUnprocessedMetadata instance.

In [5]: import music21

In [6]: rntxt = """
   ...: Note: here's a note
   ...: m1 I
   ...: """

In [7]: s = music21.converter.parse(rntxt, format="romanText")

In [8]: s.show("text")
{0.0} <music21.metadata.Metadata object at 0x118e10d00>
{0.0} <music21.stream.Part 0x118def370>
    {0.0} <music21.stream.Measure 1 offset=0.0>
        {0.0} <music21.meter.TimeSignature 4/4>
        {0.0} <music21.roman.RomanNumeral I in C major>
    {0.0} <music21.romanText.translate.RomanTextUnprocessedMetadata Note: here's a note>

We could do likewise with any ending annotations, unless you have an objection or a better idea.

mscuthbert commented 2 years ago

I just realized that music21 isn't parsing (can't really without a huge rewrite) 1st/2nd endings in Humdrum/Kern files (very strange spec for them) so it makes me appreciate how hard endings can be to get right.

I think that a metadata object and tag is probably the best place to store this -- I think I'm going to be getting out of a lot of the UnprocessedToken tags, etc. and start using Metadata for storing information about processing problems/notes/etc.

jacobtylerwalls commented 1 year ago

Fixed by #1503

cuthbertLab / music21

First/Second endings in DCML conversion #1412