Added spec for lyrics and lyric lines, with two new example docs

[adrianholovaty]

This implements basic support for lyrics:

• Events now have a "lyrics" key, which is a lyrics object.
• The lyrics object has a "lines" key, which is a list of lyric lines.
• A lyric line has a required "text" key, plus an optional "continues" boolean.

This includes two new example docs:

• Lyrics (basic)
• Lyrics (multi-line)

Note that this changes the JSON Schema to define a custom "string" type, which is not intentional. I need to tweak the docs system not to do that. I'll take care of that before merging this pull request, but in the meantime I'd like to get feedback on this basic lyrics support.

[williamclocksin]

There should be two type of continuations: hyphen (which you have specified) and melisma.

[adrianholovaty]

Just got off the biweekly co-chairs chat, in which we talked about this pull request. I'm going to make a bunch of changes:

• Rename "continues": true to "hyphen": true — to avoid possible confusion with extender lines. (Those also "continue" a note, of course!)
• Change the "lines" object from an array to a dictionary — to avoid awkwardness with having to encode skipped lines/verses as empty strings. The keys of this dictionary will be user-defined ID strings, such that lyrics in the same line will have the same ID string. In the future we'll add a way to encode the visual order of the lines (i.e., which ID string represents the top lyric line, which ID string represents the second line, etc.). By default we'll say the visual order of the lines should be alphabetical (alphabetical order of the ID strings).

This pull request deliberately doesn't deal with extender lines (melismas) yet. The current plan for those is to add a separate object to the lyric line, which would encode the fact that the lyric line begins a melisma, plus the rhythmic duration of that melisma.

[mscuthbert]

About:

Rename "continues": true to "hyphen": true — to avoid possible confusion with extender lines. (Those also "continue" a note, of course!)

One reason for explicitly noting that a hyphen will result rather than another semantic encoding like continues or mid-word = true is that multi-syllable/character words in many Asian languages should not use hyphens.

[paul-bayleaf]

If I've understood the revised proposal where each event's lyrics is an array of objects each with a user defined key then, without a specified ordering of the keys, in order to determine on which line a lyric should be placed the MNX processor has to first collect all the user defined lyric keys for the entire piece and then sort them. Only then can the lyrics' lines be established. An array, as per your original proposal, immediately gives the line position, which I think is a lot simpler to deal with. What is so awkward about empty array elements? Alternatively, if objects are to be used then rather that having a user defined key require that the key is the lyric's line number.

[samuelbradshaw]

About:

Rename "continues": true to "hyphen": true — to avoid possible confusion with extender lines. (Those also "continue" a note, of course!)

One reason for explicitly noting that a hyphen will result rather than another semantic encoding like continues or mid-word = true is that multi-syllable/character words in many Asian languages should not use hyphens.

To make sure I'm understanding correctly – is the intention in this latest change that a hyphen will always be drawn on the page if "hyphen": true, and never be drawn on the page if "hyphen": false? Or is "hyphen": true a semantic property that says the word continues, but whether or not a hyphen symbol is drawn depends on the language?

If the former, I'm worried that this change will make it harder to extract lyrics from sheet music. For example, in Korean sheet music like this: https://assets.churchofjesuschrist.org/f3/14/f3141a6cd94456a64017fd99fce71e7e384b1a98/high_on_the_mountain_top.pdf Ideally I'd be able to extract lyrics like this from the MNX sheet music:

1. 높은 산 언덕 위 깃발이 날린다 온 땅 위에 덮이니 만 백성 보네 꿀벌의 땅 평화의 곳 시온의 언덕 위에 서있네

But if we're moving away from a semantic indicator that tells me if the word ends or not, is there any way I can know where to put spaces?

[adrianholovaty]

@paul-bayleaf Thanks for the thoughts. I neglected to mention another reason to avoid using arrays, which is the possibility of lyrics above the staff. See the "Lyrics above the staff vs. below" section in issue #354. Those could be encoded as a separate "linesAbove" array, but I think it would be more elegant to store all the lyrics lines together, as the "above vs. below" decision is a bit presentational in nature.

There are also some concrete benefits in giving each line an addressable name, so that we can attach document-level metadata to them. Specific ideas for metadata about a lyric line:

• Its position (above or below the staff)
• Its order (if deviating from default, which would be alphabetical by line ID string)
• Its language (e.g., French)
• Its visual formatting (e.g., italic)

We could also have a concept of "lyric spans," where the document would define a given range of music ("bars 1-16") with some metadata about how the lyric lines should behave in the range.

Apologies, I should have written up those ideas, as I realize we talked about them in our co-chairs meeting but I didn't communicate them here.

Responding to your point about needing to calculate the number of lyrics lines in order to display the music: you'd only need to do this for a given system at a time (not the entire score). If some parts of a score have four lyric lines, but the "chorus" only contains a single line, and the chorus is on its own system, then that system would only need to allocate space for the single "chorus" line. Let me know if I'm misunderstanding the question.

[samuelbradshaw]

I have some ideas of how to make lyrics easier to extract – but I can open that as a separate issue. I don't have any objections on my end to moving forward with this pull request, so we can have a foundation for basic lyrics support. Thanks for all your work on this!

[adrianholovaty]

@samuelbradshaw OK, great to hear it. We definitely appreciate your expertise in this area!

Regarding your "hyphen" question, let me get back to you with a proper answer. @mscuthbert had the idea to rename it and I'll be chatting with him later today.

[mscuthbert]

Ah. I had not realized there were languages that did not use hyphens in musical lyrics but also used spaces when extracted. The chairs had discussed Chinese and Japanese (and Hebrew) but not Korean.

Expertise in Arabic lyric setting would also be a help if anyone has it.

[samuelbradshaw]

Responding to your point about needing to calculate the number of lyrics lines in order to display the music: you'd only need to do this for a given system at a time (not the entire score). If some parts of a score have four lyric lines, but the "chorus" only contains a single line, and the chorus is on its own system, then that system would only need to allocate space for the single "chorus" line. Let me know if I'm misunderstanding the question.

Note that the chorus doesn't always start in a new system. For example: https://assets.churchofjesuschrist.org/b4/10/b4109e8e18972270c458ac9134bfd715ad4f0c96/battle_hymn_of_the_republic.pdf System 4 starts with 3 lines, but a new line comes in mid-system for the chorus. In this case, since the number of lines goes down (from 3 lines to 1), planning for 3 lines in the system would probably be OK. But I could imagine similar cases where the number of lines goes up (from 1 line to 3) in the middle of the system – in which case you'd need to scan ahead when you're starting to layout the system, or find another way to identify which lines are relevant to which systems.

[adrianholovaty]

Just had the biweekly co-chairs chat and we decided to change this pull request a bit more:

• Instead of "hyphen", use "type". This would be "whole" (default), "start", "middle" or "end". The previous idea of "hyphen": true would now be encoded as either "start" or "middle", depending on context. This new approach means there's more data to encode, and it means there's possibility for ambiguity/bad data, but it enables a few more use cases, such as helping a graphics engine decide how close a given lyric can be rendered to another one. We would explicitly document what MNX consumers would be expected to do in each case of "bad" data (@mscuthbert created a quick rubric for this).
• Rename "lyric line" to "event lyric line" and use the term "lyric line" to refer to the entire line across the full score. We didn't previously have a term for this, and it's generally useful to have a term.

[mscuthbert]

Adrian will report more, but after discussing these corner cases, it looks like a 4-way enumeration like MusicXML's syllabic will be retained. ( https://www.w3.org/2021/06/musicxml40/musicxml-reference/data-types/syllabic/ ) -- no names determined at present. Given this, and given that mismatched <syllabic> etc. is one of the most common error conditions in music transfer, I'd like to propose a translation table for impossible combinations that consuming applications should use. Assuming the names are "start" (S) "mid" (M) "end" (E) "whole" (W) (musicxml "begin" "middle" "end" "single"), and assuming one-distance look-behind (store last value) with only the second/current value changeable:

S S -> S M     M S -> M M     E S -> ok      W S -> ok
S M -> ok      M M -> ok      E M -> E S     W M -> W S
S E -> ok      M E -> ok      E E -> E W     W E -> W W
S W -> S E     M W -> M E     E W -> ok      W W -> ok

A consequence of this table is that applications that can only encode a boolean "hyphen-follows" value can encode true/false as S + W and then (I believe) can walk through the data a second time to convert to S,M,E,W values that are consistent with that data.

[samuelbradshaw]

Expertise in Arabic lyric setting would also be a help if anyone has it.

Re: Arabic, I've seen two different styles, but I'm not sure which is preferred by native speakers.

1. Lyrics inline with the music, with the musical notation right to left, as in this book: https://www.churchofjesuschrist.org/bc/content/shared/content/arabic/pdf/language-materials/33438_ara.pdf#page=7
2. Lyrics below the music: https://assets.churchofjesuschrist.org/23/0b/230bffd9d2583a84bcc420f3ab5308dd7abcbb0e/count_your_blessings_ara.pdf

Neither has hyphens. I'm not sure if any modern notation software natively supports the right-to-left music notation style.

Languages like Thai and Khmer/Cambodian are sometimes the opposite of Korean. Whereas Korean has spaces in lyrics, but no hyphens in the sheet music, Thai and Khmer sometimes have hyphens in sheet music, but no spaces in the lyrics. Here's a Khmer example: https://assets.churchofjesuschrist.org/59/a1/59a17ff8953f52801ad77aca927ec5c6d92cf832/the_morning_breaks.pdf Lyrics: ថ្ងៃ​រះ​ឡើង​ហើយ​ស្បៃ​ង​ងឹត​ឃ្លាត មើល​ទង់​ស៊ី​យ៉ូន​ត្រ​ដាង​លា​លាត! ព្រ​លឹម​ស្រាង​ៗ​នៃ​ថ្ងៃ​រះ​ថ្មី ព្រ​លឹម​ស្រាង​ៗ​នៃ​ថ្ងៃ​រះ​ថ្មី ព្រះ​ចេ​ស្តា​ទ្រង់​ចាំង​ពេញ​ផែន​ដី។

[mscuthbert]

Thank you Samuel! Wow. What an amazing set of resources. There are cautions in using one publisher for so many different language resources (whether a pan-editorial condition overrides local practice) but I think everyone in the field has very high respect for LDS hymnal editing practices.

And thankfully lyrics are one of the few places where Mongolian does not pose especially hard or unique problems (as the world's sole(?) remaining strictly vertical only language -- unlike Japanese/Chinese etc which can be written either way.)

[samuelbradshaw]

There are cautions in using one publisher for so many different language resources (whether a pan-editorial condition overrides local practice) but I think everyone in the field has very high respect for LDS hymnal editing practices.

Thank you – agreed, I'd love to see more language examples from other publishers.

[paul-bayleaf]

"Its order (if deviating from default, which would be alphabetical by line ID string)"

Alphabetical ordering of Unicode is not a trivial matter! It is locale dependent. E.g. Swedish: z < ö German: ö < z For MNX surely the ordering must be locale independent, otherwise we lose portability. If we go down this path we will need to be very precise as to what "alphabetical" means.

Unicode® Technical Standard #10 Unicode Collation Algorithm https://www.unicode.org/reports/tr10/

[williamclocksin]

One question: It appears that the provision for lyrics assumes that all the lyric syllables on a given line are in the same font. Is there ever any call for putting one or more of the syllables on the line in a font (i.e. name, weight, posture) other than the one specified in the lyric line?

W.

On 10 Oct 2024, at 18:53, Adrian Holovaty @.***> wrote:

@paul-bayleaf Thanks for the thoughts. I neglected to mention another reason to avoid using arrays, which is the possibility of lyrics above the staff. See the "Lyrics above the staff vs. below" section in issue #354. Those could be encoded as a separate "linesAbove" array, but I think it would be more elegant to store all the lyrics lines together, as the "above vs. below" decision is a bit presentational in nature. There are also some concrete benefits in giving each line an addressable name, so that we can attach document-level metadata to them. Specific ideas for metadata about a lyric line: • Its position (above or below the staff) • Its order (if deviating from default, which would be alphabetical by line ID string) • Its language (e.g., French) • Its visual formatting (e.g., italic) We could also have a concept of "lyric spans," where the document would define a given range of music ("bars 1-16") with some metadata about how the lyric lines should behave in the range. Apologies, I should have written up those ideas, as I realize we talked about them in our co-chairs meeting but I didn't communicate them here. Responding to your point about needing to calculate the number of lyrics lines in order to display the music: you'd only need to do this for a given system at a time (not the entire score). If some parts of a score have four lyric lines, but the "chorus" only contains a single line, and the chorus is on its own system, then that system would only need to allocate space for the single "chorus" line. Let me know if I'm misunderstanding the question. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[adrianholovaty]

@williamclocksin We don't have a way to specify fonts for lyrics yet, but, yes, that's definitely something we should aim to support.