clivis stem length

[eroux]

Clivis stem length may vary according to books, squarize.py needs to be able to output different types of stem length.

[eroux]

(well, there are some incoherences I'm a bit puzzled about in this image, so it's going back to the drawing board, sorry...)

[henryso]

Could it be that the "longqueue"-ness of the line depends on the position of the bottom of the line rather than the note at the top of the line?

[eroux]

You're 100% right, I didn't realize that! Anyways, the global idea is:

• for deminutus, have same queue length as if it was not a deminutus figure
• for non-deminutus, have the queue slightly below the bottom of the second note

[eroux]

Things are not 100% decided yet, but here are some features that could be implemented already:

• longueue-ness always depends on the second note
• apply the same exception for porrectus with first ambitus of 1 as for clivis: when the second note is between lines (fgf), the queue must go beneath the line (just like clivis)
• when the second note is on a line, the queue must go slightly beneath the second note (so there is a difference with the case where the second note is between lines, which is not fully decided yet)
• deminutus (gf~) follow globally the same rule as clivis (but details are not all decided yet)
• ancus follow the same rule as clivis (third note not taken into account except in particular exceptions, details to come)
• salicus, pes quadratum, etc. follow the same rule as clivis for stem length (details to come)
• queues should be shorter for lower notes (details to come)

Options for cases where the second note is between lines (hg):

1. keep the same length as we have now (if possible a bit lower, but that's not very important)
2. cross the bottom line
3. stop the stem in the middle of the second note

One of these options will be chosen, but it's good to keep that in mind, and, if it's simple (which I doubt, but who knows), users could choose between these options.

I think several points can be discussed before having all the details:

1. first, does everyone agree with those changes? It seems the two big projects using Gregorio agree, but Gregorio should remain independent!
2. what the best way to discuss that? Here in this topic? A very complete example should be built with all these stems so that we can "draw" the correct stem length on it, but I don't know what would be the best way to do that
3. all these new glyphs for porrectus (even just with ambitus one), ancus, salicus, clivis, pes quadratum, etc. will be many, will it fit in PUA? If not, can we take other private use parts (not supported in old LuaTeX), or should we use the fuse system?

[henryso]

To answer your points directly:

1. I'm not really the person to ask about the artistic nature of things, but I am happy with these suggestions. I have always felt that (most of) the stems produced by Gregorio are too short.
2. I think it would be good to open this up to the list once the details are worked out, if only to get input (or tacit acceptance).
3. If we select exactly one of those three options, we're essentially doubling the number of all glyphs for which the second note is lower than the first. If you include the porrectus flexus, we will overrun the BMP-PUA (or get really, really close). We can certainly push into PUA-A and PUA-B, but, as we know, that slows things down significantly. If we remove the old-style fusion of the torculus resupinus flexus, we should have enough space to fit everything.

We can give the user those stem options through different fonts (like the Dominican fonts), but that's 18 fonts in total, each about 1MB in size, so perhaps the user will have to roll their chosen variety if they don't want the default or we can have a supplementary font pack or something.

[eroux]

Looking back at the font, I think we're ok in terms of number of glyphs, as we already have longqueue versions for all of them except porrectus, so that should fit easily right? I also like the idea of an additional font pack, or maybe we can provide some fonts per request (if we implement a switch in squarize.py, which seems like a good idea)?

[henryso]

We have about 1000 free glyphs. Porrectus adds 100 longqueue glyphs, porrectus flexus adds 500, fusion flexus adds 5. I think that should fit, then.

[eroux]

Good, that was close! I've created fix-803 branch on this repo so that we can work all together on this if needed. @henryso I was wondering if you could make the LongQueue depend on the second note (or previous note in case of pes quadratum, salicus, etc.) on the C side?

[henryso]

@eroux I'll try to do it this weekend.

[henryso]

0a8b40e5cd3775377e5c5ea194744cf413236105 generates "Porrectus...Longqueue" shape references, which will obviously break until they are added to the font. The "virga reversa" and "oriscus scapus" fusion glyphs will need to change as well, to lengthen their stems.

It turns out that we already have long and short varieties of those (though with virga-length stems throughout the ambitus range), which saves us 5 glyphs. On the flip side, I forgot about the flexus fusion glyphs (the ones that look like the first two notes of a porrectus, which that commit also will generate) so that adds 25 more. Finally, there are the old-style "porrectus deminutus" shapes, for another 25 more. We can still fit, but it's getting tighter.

[eroux]

Thanks a lot, this is a huge help!

[henryso]

I made the porrectus ambitus-one change. Once the fonts are updated, I'll have a better idea if it actually works.

[eroux]

Thanks a lot! Things are getting quite slowly on the font size, because the more information I get the more difficult it becomes... There are other things I'll need from the C side, but I'll tell you when things settle down

[eroux]

If you take a look at 1934 Antiphonale Monasticum p. 396 line 1, p. 347 line 7 and p. 397 line 4, you'll see that clivis deminutus with ambitus 1 can have 3 different stem length:

• one for when the second note is on a line
• one for when the second note is between lines
• one for when the second note is a or c if there is no ledger line

the first two are the same, but in some fonts they may not be, so there really are 3 different cases to handle.

I don't really know how to handle this, a few more information:

• as far as I know, this phenomena happens only with ambitus 1 in the antiphonale, but I'm afraid we may have surprises if we don't let all the possibilities available (I'm afraid some custom font will need this for other ambitus)
• what would be a proper name for the third form? we already have nothing and 'longqueue'
• let's focus on flexus first, we'll see the other shapes later

[henryso]

How about the one on page 397, line 5? That stem doesn't even reach the bottom line and contradicts the picture at the start of this issue.

Also, if the second note is a, there will be a ledger line.

Page 397 is actually pretty good. It shows four different cases.

[henryso]

The best word I can come up with is "open". An "openqueue" has nothing below it. Of course, looking at line 5 of page 397, that's not exactly an open queue, but at least it's a word that's not too long.

[eroux]

The image at the start of this issue is quite out of date now, it should be ignored (I'll edit the description).

Also, looking back at source material, this only concerns clivis and clivis deminutus with ambitus one, not the others, so no need to be that generic. Having just FlexusOneNothing.open and FlexusOneDeminutus.open would be enough in terms of font should be enough. What do you think?

I think the main difficulty though is for the C code to call these when needed. It's particularly difficult for dc, because the glyph used will be different if there is a ledger line or not. And by nature we cannot really decide this, even with convoluted TeX algorithm trying to guess from saved position values. The reason why we cannot decide (at least not in a definitive way) is that if this appears slightly after a ledger line, a user may want to keep the normal glyph, not the .open one. The problem is that I cannot see any way to do that except by adding a command in gabc, something like (dc[ll]) that would force gregorio not to take the .open version. What do you think?

[henryso]

We already have a way to suggest to gregorio that there is a ledger line: [ll:0] tells gregorio to assume there is no low ledger line and [ll:1] tells gregorio to assume there is a low ledger line. We can piggyback off of that and the same heuristic for which this was a correction.

Also, I wouldn't use .open since that is for variants. This would be more like FlexusOpenqueueOneNothing and FlexusOpenqueueOneDeminutus.

[eroux]

That would be wonderful to adapt [ll:x] indeed! Ok for FlexusOpenqueueOneNothing, looks good. The main question now is the following, which direction to go between:

• a system in which some data are coded in the C code, like for instance "if this is dc or ba clivis, use the open variant" and "for porrectus dcd and bab, use normal instead of longqueue" (that will be needed btw)
• a very generic system in which there are Openvariants for all clivis, pes quadraum, porrectus with ambitus 1, etc. and the C code has very generic code to handle it

I would tend to do the first, but I'm often wrong in this kind of decision...

[henryso]

It's hard to answer without seeing the full breadth of use cases in front of you (and even then might be difficult). I would go with the latter if there is even a small possibility that you'd want to use the "open" variant somewhere other than below the staff, and if that's the case, I would first try to see if the rule we are using is actually what's wrong before implementing the generic system.

The thing that should lean you towards the generic system is if someone wants to use the "open" glyph in some unusual place, and when asked why, the answer is something like "it looks better that way" or "I like how that shape looks there", and not something technical like "it's next to something", or "it's between something and something else", or even "in pieces prior to 1000 AD, this better represents the figure".

[eroux]

Well, considering that it took 9 years before anybody complained with the stem length, I think only very few people really care about it... And when trying to ask them what they really want, the extremely blur answer I receive makes me think that nobody will be very strong minded about that! So the first option seems quite good...

If we go in that direction, what needs to be implemented is:

• for dc, dc~ (as well as variants with an oriscus) and cqd (as well as variants with an oriscus or a quilisma), the open variant must be selected if there is no ledger line
• for ba, ba~ (and variants) and bqa (and variants), the open variant must always be selected
• for dcd, dce, etc. and variants, the lonqueue variant must not be used if there is no ledger line, and for bab, etc. and variants, longqueue variant must never be selected

There will also be rules for salicus and ancus, but these are not decided yet... I'll make the font part, if you can make the C part that would help quite a lot!

[henryso]

Is this just for ambitus one? Is ec open if there is no ledger line? Never mind, reviewing the comments, you've already answered this.

[eroux]

I think you deleted your comment about lonqueues on a, but it was relevant, I think all glyphs should follow the same rule as porrectus described above (always short for a, short for c if no ledger line)

[henryso]

I deleted it because I was off by 1. b was always getting a longqueue, and it should continue to do so. a was always getting a shortqueue, but now should get an openqueue if that applies.

[henryso]

If there is a ledger line below a d, it currently gets a longqueue, else a shortqueue. Should this behavior change with regard to this? Should it get an openqueue instad of a shortqueue? Does it differ for a single note or as the second note in a clivis?

[eroux]

That's a good question...

For the second note of a clivis, there is no change if the second note is d in the 1934 Antiphonale Monasticum (p.402 line 5 and p. 373 line 3), but I'm not entirely sure everyone will agree with this... So let's keep it normal (longueue) for now, and if there is a strong will to change, we'll provide it as an option.

But for the virga, I'd say open, as dv should be coherent with dc...

How does that sound?

[henryso]

I'm fine with the first part of your answer, but the second has a bit of nuance to it.

dv is based on the d pitch but dc is based on the c pitch. dc is either shortqueue (if there is a ledger line) or openqueue (if not). dv would be longqueue if there is a ledger line or what if there isn't? shortqueue? openqueue? something new?

[eroux]

I was thinking about openqueue

[henryso]

The logic is becoming a little bit complex now. While I will continue making the changes you request and doing some spot checks before the fonts are ready, I expect there to be a number of problems that I'll have to fix when the fonts come in.

[eroux]

Yes, sorry to take so much time! I'm struggling with fontforge right now (points don't move when asked to!)... I hope I'll be able to have something ready tonight, but I'm not really sure...

[eroux]

@henryso the fonts should generate correctly now if you want to test the C part

[henryso]

Notes for #840 and #841

The following things were changed in some way:

• greciliae.yaml was missing. I just cloned gregorio.yaml since it's most likely the same.
• squarize.py complains that VirgaLineBR is missing from gregorio, greciliae, and parmesan. I assume this is intentional, but a fix is required on the TeX side, because it uses this for glyph for the InitialConnectedVirga and FinalConnectedVirga offset cases. In the interim, I used the Virga shape for these cases. I have not corrected the bits of squarize.py that attempt to copy VirgaLineBR because I wasn't sure enough of your intent.
• It seems that Virga(Long|Short|Open)queueNothing glyphs are generated whereas, the old equivalent would be Virga(Long|Short)queue (without Nothing). I'm assuming that was an oversight in squarize.py, so I removed Nothing when generating Virga and VirgaReversa shapes.
• I found/fixed a bug generating PorrectusFlexus...Nothing glyphs.
• Virga and VirgaReversa were reversed. (gv) should have the stem on the right. I fixed it in squarize.py without renaming the base glyphs, but you might want to change something around (i.e., after my change, rvirgabase is used for Virga and virgabase is used for the VirgaReversa).
• The C code was erroneously generating PorrectusNobarLongqueue and PorrectusFlexusNobarLongqueue, which makes absolutely no sense. Fixed.
• The C code had some issues generating fusion from the openqueue shapes. Fixed.
• The C code was generating open queue for clivis/quadratum at ambitus > 1. Fixed.

Other Notes:

• I also ran into a problem because I didn't have pyyaml installed, which I suppose is now a dependency of squarize.py. While it wasn't too hard to install that, we might want to consider making the configuration file executable python code, to keep the dependencies simple.
• In the greciliae font (and presumably gregorio and parmesan), the long queues on the clivis/porrectus look like they are touching the lines. I'm guessing that's not right, but I haven't tried to fix it.
• This PDF shows my interpretation of the rules. Does this agree with your vision of the rules?
• I was also thinking that, for both long and short queues, the stems on clivis/quadratum should cross the line below the lower note at ambitus one, but that's not what I'm seeing (see the PDF). This appears to be the case in Solesmes books, but maybe I'm misinterpreting it.

[eroux]

Thanks a lot! I'll be fixing the problems you spotted this morning, a few other points:

• about a pure python config file, I think it's a good idea, allowing more interesting configuration
• about your pdf, I think av and cv (with no ledger line) should be open, the rest looks fine, the errors are mine

[eroux]

I've changed yaml to json (as support seems to be native), what do you think?

I've also started to actually tune queues a little, it seems more reasonable than what we had before, but still many details (and bugs) to work on!

[henryso]

I'm fine with json. Native support clinches it.

In reviewing the font generation error messages, I found an issue with the the ascending oriscus scapus fusion glyphs, which I've pushed/merged into the branch.

The queues do look a bit better, but I agree they need more tuning.

[eroux]

thanks a lot for the bug! I'm almost done with a first serious attempt (which will need to be validated by the Antiphonale Monasticum and Solesmes), but I need a few small modifications in the code:

• glyphs with ambitus 1 should call the long version when the note is d, eg ed (this is the one I'm not 100% sure, but AM 1934 does this)
• glyphs with ambitus > 1 should call the long version when the note is d or b (eg hd, fb)
• porrectus should call their long/short version according to the first note, not the second (rules are the same as the virga)

Would that be easy for you to make these changes?

[henryso]

I'll try to make these changes tonight, but I can't guarantee that I will complete them as I have an activity tonight.

[henryso]

I think I made the changes your requested, but I admit it's getting a little confusing. Here's the updated PDF.

[eroux]

Thanks a lot! There are still a few things to be dealt with... could you send me the corresponding gabc file so that I can modify it directly?

[henryso]

I pushed the test that generates that PDF into gregorio-test under the fix-803 branch.

[eroux]

The current state starts to look fine. I'm supposed to get an answer from the Antiphonale Monasticum project soon, but I think you can start to review the default stem length schema. You can use this gabc to review most of the different lengths. Please write all your remarks, Gregorio doesn't have to do exactly the same as the Antiphonale Monasticum project or Solesmes, so we can add another separate schema.

[eroux]

I got a confirmation on two points on the C side:

• porrectus should always follow virga: dcd and bab should be short
• for clivis (and everything determined in a similar way), when the second note is on d, the queue should be open

@henryso do you think you could make the change?

[henryso]

Should dcd be short regardless of whether there is a ledger line?

[eroux]

Sorry, no, if there's a ledger line, it should be long

[eroux]

A few more changes if you have time:

• it would certainly be more understandable to switch "long" and "short" on second clivis (second only), as it corresponds to their actual length (in the font gf has a shorter stem than hg)
• [ll:0] and [ll:1] don't seem to be working, example g(ed~[ll:0]) outputs a \GreCPFlexusLongqueueOneDeminutus and g(fe~[ll:1]) outputs a \GreCPFlexusOneDeminutus
• [ll:0] on a glyph which stem length is determined by a note on d (dv[ll:0], ed[ll:0], etc.), c, b or a should generate an Openqueue version when available

[henryso]

What do you mean by "second clivis"? Ambitus one clivis?

[henryso]

I think [ll:0] and [ll:1] are working, and you are being thrown off by the "confirmation" changes not being implemented. After those changes, it looks like this:

ed[ll:0] produces \GreCPFlexusOpenqueueOneNothing, ed[ll:1] produces \GreCPFlexusLongqueueOneNothing (again, AFTER the changes).

[henryso]

I think it looks pretty good based on your test file. My only problem is with ba (open queue) and aqb (also open queue):

\GreCPFlexusOpenqueueOneNothing

versus

\GreCPPesQuadratumOpenqueueOneNothing

Perhaps this is related to what you meant by "second clivis"?

[eroux]

Thanks, I meant ambitus one indeed

[eroux]

I've pushed some fixes, inluding the one for open pes quadratum, thanks for spotting it!

I've updated https://gist.github.com/eroux/9ff5147101525415ad3b too, it's meant to be tested with

\grechangeglyph{Porrectus*}{*}{.alt}

One final change that will be needed is for alternative porrectus deminutus, I now generate all PorrectusLongqueueXYDeminutus and PorrectusLongqueueXYDeminutus.alt (Deminutus only), could the C code reflect that?

Also, what I meant with my ambitus one short-long inversion is the following:

• open versions, one-note versions (virga) and versions with ambitus > 1 are not concerned
• gf is considered long because the second note is one a line, while in the font, it has a shorter queue than hg, which is considered short, so I think (maybe) we can switch their "longness". I would revert them into the font, and you would in the C code, this would just be an internal change...

What do you think?