Add text spacing option to {start_of_textblock}

[gwyndaf]

It might be a useful enhancement to add a textspacing option for {start_of_textblock}, if that's feasible.

Although it's possible to set textstyle to match other font styles in use, it seems to use a default spacing, which I'm guessing is the pdf.spacing.lyrics value.

{start_of_textblock-guitar: textstyle="tab" anchor="page" x=100% y=100% scale=95%}
lets me fit a short tab into some free space bottom right of the page, except that it's spaced more widely than tab created with {start_of_tab}.

It's probably too complex to infer spacing.tab from textstyle="tab", so maybe simplest/clearest just to have a manual option for line spacing?

Since there are other spacing considerations for positioning the whole block on the page, spacing could be ambiguous name, so textspacing might more clearly refer to spacing of text within the block.

[sciurius]

I recently fixed a problem that the textblock did not take the correct (text) spacing into account when calculating the dimensions of the block. Nevertheless the general appearance should have been okay. The attached example shows the same font and spacing for the textblock and the tab.

(The textblock is missing line 4, that's the bug I fixed).

[sciurius]

The technical term is leading. As of 6.050_088 you can add a leading property to a font definition. Note that this will be interpreted as a factor to the font size, not an absolute value. Currently the only place where leading is taken into account is in the textblocks. So now you can do fancy things like:

pdf.fonts.fancy {
    file: Inkpen2Script.ttf
    size: 8
    leading: 1.2
}

And use it in a textblock:

{start_of_textblock textstyle="fancy"}
...
{end_of_textblock}

[gwyndaf]

Hmm, interesting. With your tb.cho.txt sample and ChordPro defaults, I don't get the same spacing between lines, as you describe. Instead, textblock lines are much tighter than regular tab lines. tb.pdf

Might that be because the default tab font, 'Courier', references system Postscript fonts (or alternatives), which often vary from system to system? I have both Tex Gyre Cursor and Nimbus Mono PS available as Courier substitutes, and the first gets used by default. Although the glyphs are near-identical, those in Nimbus Mono sit higher in the line, so possibly the two fonts have different built-in leading, which affects output and calculations? Leading's a concept I know, but more from a design/typography point of view, rather than font technologies and measurement standards.

For clarity (to avoid PDF font substitution confusion), this screenshot is how the above PDF manifests on my system:

On the other hand, using my own default config reverses the effect and the textblock tab is looser: tb (Guitar) git.pdf

In terms of pdf.spacing values, my config's lyrics is 1.2 (same as default) and tab is 0.9 (default is 1), so I'd have expected a similar sort of effect, but get further confusion! I'll need to explore this further, but am I right in assuming that texblock line spacing (not character leading) defaults to the lyrics value (1.2)?

In practical terms, I've successfully used fonts.tab.leading to make textblock tab appearance match that of regular tab (my original aim). With trial and error, I find it needs a value of 0.69 to achieve the same affect as standard tab with with pdf.spacing.tab 0.9, using my own configs. I can't figure out why it needs that value. Even if default line spacing for texblock is 1.2, and character leading is applied on top of that (multiplied), I'd expect to need a value of 0.9 / 1.2 = 0.75, so it's not clear how the values relate. Perhaps the calculations involve more than simply spacing and leading values, and - again - possibly the font's built-in leading is a factor?

Maybe related, built-in leading might affect calculation of textblock size. My tighter leading setting leads to the bottom characters being cut off: All other characters are output in full, so it doesn't seem to be caused by the small leading value in itself, but maybe it's down to the calculated block height being very slightly too small.

[sciurius]

Several things playing roles here...

With trial and error, I find it needs a value of 0.69 to achieve the same affect as standard tab with with pdf.spacing.tab 0.9, using my own configs.

It's easy once you have been bitten by this a couple of times ☹. When asked for the font height, some fonts return a value that includes the optimal line spacing. The distance between baselines is the font size (corresponding with spacing = 1). Courier and GNU FreeMono reports the baseline distance as text height, while Tex Gyre Cursor reports approx 1.2 times the baseline distance as text height.

In a tab, ChordPro uses the fontsize to calculate the line advance, in a textblock it uses the reported font height.

This is what I get with the following settings:

pdf.fonts.tab {
    file: texgyrecursor-regular.otf
    size: 10
    leading: 0.75
}
pdf.spacing.tab: 0.9

Note that in a textblock, leading overrules the spacing.

The bottom cutoff should be gone in 6.050_089.

tb.zip

[gwyndaf]

Thanks Johan. That clarifies things a lot, knowing that spacing and leading are based on different font metrics. I'll need to think through the implications, but that seems to point the way to understanding equivalent values for a given font.

I can see that testing with the default tab font could actually confuse matters, because many systems substitute an alternative, which could yield different metrics on different systems, depending which font presents as 'Courier'.

In production use, that's not an issue for me because I use a specific font (IBM Plex Mono) for tab. Since spacing 0.9 and leading 0.69 look the same, I'm guessing that font's probably got a 1.3 multiplier between text height and default line height. I'll check that, once I can get my head around font metric tools!

I can confirm that bottom cutoff has gone in _089.

[sciurius]

There are a lot of surprises. E.g. fonts that do (or don't) take accents into account:

As you did with IBM Plex Mono, you will have to do the finetuning with specific fonts.

[sciurius]

Another significant difference between using the actual text height to space lines, or the font size * spacing:

My intention is to (some day) switch lyrics spacing to incorporate the actual text height. What do you think?

[sciurius]

Note that in a textblock, leading overrules the spacing.

On second thought, I'm not sure this is right. It is probably more intuitive if the leading is multiplied by the spacing. For example, assume the grid font has an internal leading of 1.15, a user specified pdf.fonts.grid.leading:0.8 and pdf.spacing.grid:1.2 the effective distance between the baselines at font size 10 would become 10 * 1.15 * 0.8 * 1.2 = 11.04. Does that make (more) sense?

[gwyndaf]

I think anything that combines both 'line spacing' and 'leading' values is potentially confusing because it provides two different measurement for what's essentially the same purpose (from a user point of view). A sensible long-term aim might be to have only one such measurement throughout (though I appreciate they represent different functions within ChordPro).

In your example, I think the textblock behaviour is probably more in line with what users would expect. That is, the line spacing used in a particular instance reflects the actual type height in use (including character-level markup), whereas the lyrics example seems to base spacing on line-level type size (e.g. {textsize}), with no further accommodation for character-level adjustments.

I know no software that allows for both line spacing and leading values. For instance, MS Word and LibreOffice Writer tend to work with 'line spacing' (as a paragraph property) as either a specific height (in points) or more usually a proportion, which I assume relates to the font's built-in line spacing, but don't know for sure.

On the other hand, Adobe InDesign works with 'leading' (as a character property), as a point size, which is more common for experienced designers. Although there is an 'auto' leading value, Adobe docs suggest that's a standard 120% of type size, and doesn't use the font's built in line height. https://helpx.adobe.com/si/indesign/using/leading.html.

It's also a bit confusing because, historically, leading in physical typesetting was literally a row of lead/metal added above/below the relevant type, so was additional to the type size. However, most computer typesetting refers to 'leading' as the total height from baseline to baseline, and is the sense most designers today will understand. That is, in practical usage, 'leading' and 'line spacing' are essential the same.

I also think the term 'line spacing' may remain clearer to users without a design background. 'Leading' is also an ambiguous homonym in English, and users who aren't designers or native English speakers might think it means 'coming before' (e.g. space to the left).

I think there are two different questions here: 1) how ChordPro internally obtains and calculates the space from one baseline to the next, and 2) how such a measurement is named for user configuration purposes.

Here's a suggestion: could the term spacing be used in two contexts, i.e. as a legacy line attribute like pdf.spacing.lyrics and as a forward-looking character/font attribute, e.g. pdf.fonts.lyrics.spacing? If both follow the same convention (multiple of text size), their product would give the final configured value, which would then be multiplied by the font size actually in use (e.g. taking into account any markup)?

I'll admit I'm still struggling to understand exactly what ChordPro's 'leading' and 'line spacing' measurements represent, in terms of font measurements (traditional or computer). I've just starting using font-line as a useful way to quickly check font line metrics, but can't relate ChordPro's sense of 'leading' and 'line spacing' to standard font metrics, like

$ font-line report IBMPlexMono-Regular.ttf 
=== IBMPlexMono-Regular.ttf ===
Version 2.1
SHA1: 10dd78242e14dc964636faf41ffd2d209a676286

::::::::::::::::::::::::::::::::::::::::::::::::::
  Metrics
::::::::::::::::::::::::::::::::::::::::::::::::::
[head] Units per Em:   1000
[head] yMax:           1119
[head] yMin:          -251
[OS/2] CapHeight:      698
[OS/2] xHeight:        516
[OS/2] TypoAscender:   780
[OS/2] TypoDescender: -220
[OS/2] WinAscent:      1025
[OS/2] WinDescent:     275
[hhea] Ascent:         1025
[hhea] Descent:       -275

[hhea] LineGap:        0
[OS/2] TypoLineGap:    300

::::::::::::::::::::::::::::::::::::::::::::::::::
  Ascent to Descent Calculations
::::::::::::::::::::::::::::::::::::::::::::::::::
[hhea] Ascent to Descent:              1300
[OS/2] TypoAscender to TypoDescender:  1000
[OS/2] WinAscent to WinDescent:        1300

::::::::::::::::::::::::::::::::::::::::::::::::::
  Delta Values
::::::::::::::::::::::::::::::::::::::::::::::::::
[hhea] Ascent to [OS/2] TypoAscender:       245
[hhea] Descent to [OS/2] TypoDescender:     55
[OS/2] WinAscent to [OS/2] TypoAscender:    245
[OS/2] WinDescent to [OS/2] TypoDescender:  55

::::::::::::::::::::::::::::::::::::::::::::::::::
  Baseline to Baseline Distances
::::::::::::::::::::::::::::::::::::::::::::::::::
hhea metrics: 1300
typo metrics: 1300
win metrics:  1300

[OS/2] fsSelection USE_TYPO_METRICS bit set: False

::::::::::::::::::::::::::::::::::::::::::::::::::
  Ratios
::::::::::::::::::::::::::::::::::::::::::::::::::
hhea metrics / UPM:  1.3
typo metrics / UPM:  1.3
win metrics  / UPM:  1.3

[gwyndaf]

Having thought this through further, it seems like the ChordPro leading value reflects the font's built-in line spacing (baseline to baseline), so is similar to wordprocessor 'line spacing'. On the other hand, spacing seems based on the font size alone, so perhaps more like InDesign's 'leading', which could add to confusion.

It occurred to me that your examples essentially reflect fixed (absolute) or proportionate (relative) line spacing. Example A essentially spaces lines in proportion to the actual type size used within a line (so increases to accommodate larger type). Example B is essentially fixed spacing, which remains the same regardless of actual type size in the line.

Both seem to be valid use cases from a design point of view. For simple legibility, proportionate spacing (Ex. A) ensures that lines accommodate whatever text size is actually used.

However, fixed line spacing can be useful for visual flow from line to line, to ensure a consistent read from one line to the next. For instance, if there are slight type changes within a line, it might be preferable that the spacing 'rhythm' remains consistent through a verse, even when type size/metrics change slightly.

An example might be alternating typefaces within a line or verse (or possibly even alternating Roman/italic variants), for a 'call and response' or alternating vocal parts. If the different fonts contain different built-in spacing, overall flow could be inconsistent so, in that instance, fixed spacing could be useful. For everyday purposes, though, a proportionate scaling factor seems simplest.

I notice the Pango line_height value allows for both relative and absolute values. I realise that isn't the same as ChordPro markup, but might indicate that the facility exists to specify either relative and absolute spacing values for line spacing (i.e. current leading value).

[gwyndaf]

The whole question is further complicated because there are three(?) standards for font metrics, notionally/historically related to different operating systems. This seems a clear overview.

Core question is: what exactly is the 'font size' for ChordPro?

As far as I can see, the typo (OpenType) and hhead (Apple legacy) standards include values for ascender, descender and line gap (true leading) values, while the win standard (Windows legacy) provides for only ascender and descender values.

However, Google fonts guidance suggests that LineGap should be 0 for the hhead standard, because it's not applied consistently across systems.

So, for hhead and win standards, line spacing is effectively built into the font size. With the typo standard, it remains an additional value. That's borne out in my IBM Plex Mono metrics (above, and below from FontForge), where each standard totals 1300 units in height, but only typo separate that into 1000 for font and 300 for line gap/leading.

It's a niche topic (and the first time I've delved into this much depth), but it seems the general trend is toward the typo standard. Although that seems to be the current Windows spec, it seems like there's a 'Really use Typo metrics' bit in font files, which can force legacy Windows applications to use typo rather than win metrics.

So, with ChordPro's systems for spacing and leading, is the difference in spacing because they read different metric standards, or because they use different values from the typo metrics, i.e. spacing reads only font height (ascent - descent), while leading reads line height (ascent - descent + line gap)?

[sciurius]

Thanks for you valuable input. You provide feedback faster that I can respond to ☺

My typesetting backgrounds are classic (yes, with lead glyphs) and TeX. It is unfortunate that many (more) modern tools choose such hard and differing ways to obtain elegant results. For example, early typewriters had a single, monospace font, usually some form of Courier. There were (usually) three line spacings: single, 1.5 and double.

ChordPro has two 'modes' of typesetting lines, I'll call these (most likely incorrectly) baseline based and metrics based. The vertical spacing we are currently discussing is the baseline-to-baseline distance (BTBD).

For ChordPro (lyrics) I use a BTBD of the font size * pdf.spacing. As you mentioned this makes spacing the same 'rhythm' (dutch typographical term is 'stramien'). In particular, when you have columns the lyrics lines and the chord lines will be at the same heights. For book printing this is also important because the text of a page will line up with the text at the back of the page that shines through. Note that for this spacing, none of the head, win and typo metrics are used. Well, not really -- the ascender is used to calculate the distance from the top to the baseline. For most fonts this will be the Ascender from the hhea table.

I think for most purposes font size * pdf.spacing is the best, since the BTBD stays the same even when you use a different font (or variation) with the same font size.

The situation is different when it comes to textblocks. Textblocks have no associated pdf.spacing.

TBC later.

[sciurius]

So probably the best first shot is this: when you add textstyle="tab" to a textblock, ChordPro applies the font definition for tab as well as pdf.spacing.tab. I think this is sufficient to solve your original question. You can try it in 6.050_091. Do not forget to remove the leading from the pdf.fonts.tab.

This will give us some more time to think about leading and spacing... A horrible mess...

[gwyndaf]

Ah, ok, I can see how textblock line spacing now changes to reflect changes in pdf.spacing.tab in 6.050_091.

However, it doesn't fix the discrepancy in spacing between normal and textblock layouts. So texblock tab using IBM Plex Mono spaces wider (~1.3 times) than standard tab, while default config with Courier (substitute) font spaces tighter in textblock.

Not a problem, though. I notice I can still specify fonts.tab.leading, so I can tweak that to get both regular and textbox tabs looking the same. It seems like font.tab.leading supersedes the value from spacing.tab, which seems sensible. It could be confusing (and maybe unnecessary?) combining/multiplying the two values.

Also, a new error in _091 with {start_of_textblock-guitar textstyle="tab" scale=95%} is:

Can't use string ("95%") as an ARRAY ref while "strict refs" in use at /media/data2/Shared/Music/Technology/ChordPro/git/chordpro/script/../lib/ChordPro/Output/PDF.pm line 1960.

No urgency on my part. I've no immediate production need for this, and know that font internals are rarely straightforward!

[gwyndaf]

A bit more empirical testing with Roboto Mono, which shows similar behaviour to IBMPlex Mono (larger textblock spacing). In this case the discrepancy seems close to a ratio of 1.17 (textblock:regular spacing).

That seems interesting, because the ratios reported for Roboto Mono are different for each metric standard. 1.17 happens to be the reported ratio for hhea. That information may or may not give you some insight.

$ font-line report RobotoMono-Regular.ttf 
=== RobotoMono-Regular.ttf ===
Version 2.002; 2015; ttfautohint (v1.3)
SHA1: bf3830d87a813003af577f96d5f580c5d6959fb0

::::::::::::::::::::::::::::::::::::::::::::::::::
  Metrics
::::::::::::::::::::::::::::::::::::::::::::::::::
[head] Units per Em:   2048
[head] yMax:           2163
[head] yMin:          -555
[OS/2] CapHeight:      1456
[OS/2] xHeight:        1082
[OS/2] TypoAscender:   1536
[OS/2] TypoDescender: -512
[OS/2] WinAscent:      1946
[OS/2] WinDescent:     512
[hhea] Ascent:         1900
[hhea] Descent:       -500

[hhea] LineGap:        0
[OS/2] TypoLineGap:    102

::::::::::::::::::::::::::::::::::::::::::::::::::
  Ascent to Descent Calculations
::::::::::::::::::::::::::::::::::::::::::::::::::
[hhea] Ascent to Descent:              2400
[OS/2] TypoAscender to TypoDescender:  2048
[OS/2] WinAscent to WinDescent:        2458

::::::::::::::::::::::::::::::::::::::::::::::::::
  Delta Values
::::::::::::::::::::::::::::::::::::::::::::::::::
[hhea] Ascent to [OS/2] TypoAscender:       364
[hhea] Descent to [OS/2] TypoDescender:     -12
[OS/2] WinAscent to [OS/2] TypoAscender:    410
[OS/2] WinDescent to [OS/2] TypoDescender:  0

::::::::::::::::::::::::::::::::::::::::::::::::::
  Baseline to Baseline Distances
::::::::::::::::::::::::::::::::::::::::::::::::::
hhea metrics: 2400
typo metrics: 2150
win metrics:  2458

[OS/2] fsSelection USE_TYPO_METRICS bit set: False

::::::::::::::::::::::::::::::::::::::::::::::::::
  Ratios
::::::::::::::::::::::::::::::::::::::::::::::::::
hhea metrics / UPM:  1.17
typo metrics / UPM:  1.05
win metrics  / UPM:  1.2

[sciurius]

FYI: https://github.com/PhilterPaper/Perl-PDF-Builder/issues/213#issuecomment-2247352369

[sciurius]

In 6.050_092 I have added a textblock property textspacing. You can set it to a factor for the spacing, in your case 1. Its default value is the keyword flex, which is basically the actual width of the line times pdf.spacing.xxx for known text styles xxx, defaulting to 1.

[gwyndaf]

Great, I've got 6.050_093, which seems to work nicely, although I don't fully understand the calculation basis.

In practice, a textspacing=0.9 value makes the texblock tab spacing match that of regular tab (which uses pdf.spacing.tab=0.9. Interestingly, that value gives textblock and regular tab the same spacing regardless of whether the tab font is IBM Plex Mono, or Roboto Mono, or default (Courier-like). This seems nice and intuitive, as it suggests the same result regardless of font.

I think there might be some knock-on effect on the calculated textblock size. The first two fonts mentioned above end up with a larger box than the content, while the default font ends up with a cut-off box: Textblock+tab (Guitar) git.pdf Textblock+tab_defaults.pdf

I'm not sure if it's a useful observation, but it seems like I get oversized boxes when using TrueType fonts and undersized when using a (presumed) Postscript one.

Thanks for link, which I'll read and attempt to digest :)

[sciurius]

For the sake of testing, can you add the sources + configs?

[gwyndaf]

Ah, sorry. I'm getting sloppy.

Source: Textblock+tab.cho.txt

For my custom config, --print-final-config gives Custom_config.json

However, I notice that it hasn't fully included the roboto config given as an include, so that's here: roboto.json. I'm pretty confident the roboto.json config has been processed for output, because the PDF uses IBM Plex Mono and Roboto font families, as expected.

The default/Courier PDF simply uses -X config defaults, plus pdf.spacing.tab=0.9

[sciurius]

chordpro -X Textblock+tab.cho --def pdf.spacing.tab=0.9 gives warnings:

Argument "-30px" isn't numeric in addition (+) at /home/jv/src/ChordPro/script/../lib/ChordPro/Output/PDF.pm line 2051.
Argument "-80px" isn't numeric in subtraction (-) at /home/jv/src/ChordPro/script/../lib/ChordPro/Output/PDF.pm line 2052.

Didn't you get any?

[gwyndaf]

Yes, I got the same.

I didn't mention them because they seem to relate only to positioning of the floating textbox, not line spacing, so a little off-topic.

Also, textbox positioning was as expected despite the warnings, so I thought they're probably just a minor loose end.

[sciurius]

Can you try _094?

[gwyndaf]

That's a closer fit of textblock to the content, but not quite exact.

Default/Courier looks fine: Textblock+tab_default.pdf.

But, custom config with IBM Plex Mono cuts off at the bottom baseline, losing descenders: Textblock+tab_custom.pdf

Also, not really an issue, but noting a little extra space 1) at bottom, below descenders with Courier, and 2) at top, above ascenders with IBM Plex. I'm guessing that's either down to different internal space within those fonts, or different handling of Postscript and Truetype. I don't think these are new, but I wasn't looking so closely before.

[sciurius]

I may have nailed it in 6.050_096...

[gwyndaf]

Fantastic. That looks great with custom IBM Plex and Roboto mono fonts, as well as Courier default. Textblock+tab_IBMPlex_096.pdf Textblock+tab_Roboto_096.pdf Textblock+tab_default_096.pdf

The TrueType fonts still result in very small space at top/bottom of textblock, but it's negligible and only noticeable by design pedants! I think eliminating that totally (fitting to ascender/descender) wouldn't be a good use of time.

I've not yet made much use of textblock and image features, so will continue exploring and trying to break them :)

[gwyndaf]

I've also eliminated the position-related warning messages, like Argument "-30px" isn't numeric in addition (+) ...

User error: I'd specified px units rather than numerical values only, which I realise are in points (not pixels) anyway.