ArtifexSoftware / urw-base35-fonts

Repo for URW++ base 35 font set
Other
95 stars 17 forks source link

Ligatures break metric compatibility #31

Open hvdijk opened 3 years ago

hvdijk commented 3 years ago

Nimbus Mono is supposedly metrically compatible with Courier, but Courier displays 'ff' as two separate 'f' characters, Nimbus Mono prints it as a ligature that takes up the same space as a single 'f' character. Likewise for other ligatures. When Nimbus Mono is used as a substitute for Courier on the assumption that they are metric-compatible, it breaks layouts.

Should the ligatures be disabled (except when they are specifically requested, e.g. when U+FB00 ('ff') is used), or should the font instead not be treated as metric-compatible? If you prefer to leave the ligatures as they are, please let me know and I will instead file bugs against those places that list the font as metric-compatible.

chris-morgan commented 2 years ago

ff, ffi, fi, fl, ffl. These should strongly not ligate by default, it’s obviously wrong for a monospaced font.

tomty89 commented 2 years ago

should the font instead not be treated as metric-compatible

The whole point of this font set is to serve as a replacement of the PS L2 core 35 fonts, and Courier is one (well, four) of them.

@chris-liddell Any opinion (from you or Artifex) on this? Could we at least get to choose which variant (with or without lig) to use? (I mean like variant of font file, not relying on fontfeature whatsoever in fontconfig.)

chris-liddell commented 2 years ago

Sorry for taking so long to respond.

Basically, we've tried to refer to URW++ to this (and the other issues on here), and I was planning to reply here when we had a response from URW++. But they haven't gotten back us, and honestly, I forgot.

I assume this is relates to the OpenType/Truetype version of the font.

tomty89 commented 2 years ago

Well only the OTFs. I'm not even sure if TTF support ligature anyway. I mean, at least it doesn't even seem like you can have it explicitly enabled with the CSS font-variant-ligatures: common-ligatures;. I've tested that with both the TTFs of Nimbus Roman Regular and Nimbus Mono PS Regular. But common-ligatures and no-common-ligatures enables and disables the ffi ligature respectively in both of their OTFs. The former is the default in both Firefox and Chromium. Not sure if it's a de facto or de jure standard though.

FYR the current Nimbus Sans Regular OTF on the other hand does NOT have ~(the)~ ligature support ~(that it shouldn't have)~.

tomty89 commented 2 years ago

Hmm actually according to the tech specs on MyFonts, Helvetica from Linotype does support liga, while Courier LT also from Linotype and Courier from Adobe do not.

Also, both Nimbus Sans and Nimbus Sans L on MyFonts appears to support liga as well.

On the other hand, there's a Courier PS from Monotype that supports liga like Nimbus Mono PS does. (Are they sort of related? Given that URW is now owned by Monotype, and the "PS"...)

It seems that the best alternative available is GNU FreeFont. FreeSans supports liga while FreeMono does not. (As for Tex Gyre, both Heros and Cursor supports liga...although there's no such thing as FreeSans Narrow, apparently)

tomty89 commented 2 years ago

And the Nimbus Sans Narrow in this repo supports liga while the Nimbus Sans does not...

meh

test.html.gz

chris-liddell commented 2 years ago

Well, there is an argument that they are described as metric compatible, not fully compatible fonts. By the looks of it, disabling ligature substitutions would produce metric compatible results.

But, frankly, we're not the source of these fonts, and we can't do much more than pass on the problems to URW++. We don't have that kind of font expertise in-house.

fabiangreffrath commented 2 years ago

Isn't there any fonttools or gftools command that removed ligature definitions from a font file?

ghost commented 2 years ago

I'm not even sure if TTF support ligature anyway.

It's true, TTF and T1 fonts don't feature ligatures. TTF also doesn't support cubic bezier splines, so they render slightly worse (or require much larger glyph descriptions to get an indistinguishable quality). Type 1 and OTF do support cubic splines and use them here.

ghost commented 2 years ago

Isn't there any fonttools or gftools command that removed ligature definitions from a font file?

https://adobe-type-tools.github.io/afdko/ https://github.com/fonttools/fonttools https://fontforge.org/ All have Debian packages.

That said, it's still tricky. AFDKO barely allows such edits. FontForge deliberately recompiles the entire font from scratch and applies a number of changes to align with its taste of good quality.

To list ligature sets, use

cd fonts
afdko spot -F NimbusMonoPS-Regular.otf
afdko spot -F NimbusMonoPS-Regular.otf | grep "'liga'"  # => Set 14, 15, 16.

To edit a font, convert it to XML, edit it, then convert it back to OTF:

cd fonts
ttx NimbusMonoPS-Regular.otf
# - - - edit .ttx file here - - -
ttx NimbusMonoPS-Regular.ttx

Ligatures aren't hard to find and remove, just search for "liga" or "14" (including quotes). However, there are two catches:

  1. <ScriptList> (listing font features) have pointers into <FeatureList> (listing lookup tables, some of which are 'liga'), which in turn have pointers into <LookupList>. All three structures have to be edited to avoid stale pointers.
  2. List indexes get ignored during re-compilation. Once an entry is removed, all list items behind these entries change their list position. As pointers into this list do not change, these pointers become invalid. This makes manual editing really hard and some editor recognizing and adjusting these pointers almost mandatory.

That said, I'm also researching yet another strategy.

fabiangreffrath commented 2 years ago

That said, I'm also researching yet another strategy.

Hm, as a last resort we could still fall back to packaging the TTF variant if its feature set aligns more closely to the original T1 fonts.

ghost commented 2 years ago

TTF and T1 formats don't have such feature sets, querying these variants with afdko spot -F comes up just empty.

The OTF format has more features than just ligatures. There's also a mechanism to replace 1/2 with ½ (and many similar) somehow. These are not ligatures, so far I have no idea how to take advantage of these features, or whether they should be removed from a monospaced font as well. Unlike with ligatures, typing 1/2 in a generic text editor doesn't replace anything.

fabiangreffrath commented 2 years ago

The OTF format has more features than just ligatures.

AFAIUI, in this specific case the OTF fonts aren't supposed to have any more features than the T1 fonts they are meant to replace. Anything else will be another bug in the OTF fonts.

ghost commented 2 years ago

Alright, Nimbus Mono fonts should be fixed in the attached commit. Instructions on how it was done in the commit message. I verified the result with all tests I could imagine, please test yourself with what you can imagine :-)

0001-Remove-ligatures-from-NimbusMonoPS.patch.gz

Please apply as following, this would save me all this pull request mess.

gzip -cd 0001-Remove-ligatures-from-NimbusMonoPS.patch.gz | git am

For verification after applying the patch:

md5sum fonts/NimbusMonoPS-Regular.otf fonts/NimbusMonoPS-Bold.otf \
  fonts/NimbusMonoPS-Italic.otf fonts/NimbusMonoPS-BoldItalic.otf
9f420300d69ce4779d978c45cfa50397  fonts/NimbusMonoPS-Regular.otf
ebf6cf0c8bd4eacc9f975c86e3517132  fonts/NimbusMonoPS-Bold.otf
61a8224187c71186fc12791e9185882a  fonts/NimbusMonoPS-Italic.otf
af2871b518fbd589ec7ec9bbe1988d2f  fonts/NimbusMonoPS-BoldItalic.otf
ghost commented 2 years ago

in this specific case the OTF fonts aren't supposed to have any more features than the T1 fonts they are meant to replace

If one wants T1 fonts, it's perfectly fine to use these T1 fonts. T1 even uses the higher quality cubic bezier outlines. Nimbus Mono improved a lot over the years. The version committed first in this repo had just 233 glyphs, now there are 855 glyphs. Some of these improvements took advantage of technologies which older formats like T1 and TTF don't support.

The issue opener asked for metric compatibility with Courier. That's what it is now.

fabiangreffrath commented 2 years ago

@merchantsedition Instead of providing a binary patch, could you probably recite the procedure used to create the patches fonts here instead?

ghost commented 2 years ago

It's a text patch, just compressed for convenience. This is what it writes about the procedure used:

There were 6 ligatures, 'ffi', 'fi', 'ff', 'ffl', 'fl' and 'Nr.'.  Not all for
every script (language/environment).

This applies to OTF fonts only, as T1 and TTF formats don't support ligatures
at all, so there can't be something to remove.

The challenge here was that FontForge compiles an edits fonts just fine, but
always compiles everything from scratch on saving ('Generating').  This
changes a lot more than wanted, e.g. originator texts.  It also drops table
'PCTL', which FontForge doesn't recognize.

Fonttools' 'ttx' can decompile a font to XML and compile it again, (almost)
unchanged.  However, indexes into table lists are informational, only.  Which
means, after deleting an element from a list, pointers to elements following
the deleted ones in the same list become invalid.  In principle it's possible
to correct all this by hand-editing, but it's error prone.  Removing ligatures
requires changes in lists <LookupList>, <FeatureList> and <ScriptList>.

Considering all this, the strategy used is as following:

- Use FontForge to remove these ligature entries.
- Use Adobe Font Development Kit for OpenType (AFDKO) tool 'sfntedit' to move
  table 'GSUB' (which contains these ligature definitions) from the FontForge
  edited version to the original version.
- Use Fonttools' 'ttx' and AFDKO's 'spot' to verify all this.

  # Install prerequisites.
  sudo apt-get install fontforge afdko fonttools
  # Make backup.
  cd fonts 2>/dev/null
  cp -p NimbusMonoPS-Regular.otf NimbusMonoPS-Regular-org.otf
  ttx NimbusMonoPS-Regular-org.otf
  #
  # Identify the target.
  afdko spot -F NimbusMonoPS-Regular-org.otf
  afdko spot -F NimbusMonoPS-Regular-org.otf | grep "'liga'"
  # ==> LookupLists 14, 15 and 16 should be removed.
  #     Not seen here: list 14: 'ffi' 'fi'
  #                    list 15: 'ff' 'ffl' 'fl'
  #                    list 16: 'Nr.'
  #
  # Remove ligatures.
  fontforge NimbusMonoPS-Regular.otf
  # 1. Menu -> Element -> Font Info... -> Lookups -> tab GSUB:
  #    - Delete all (3) 'liga' elements.
  #    - Click 'OK'.
  # 2. Menu -> File -> Generate Fonts...:
  #    - Drag the dialog larger to see all choices.
  #    - Top left choice, select 'OpenType (CFF)'.
  #    - Enter filename 'NimbusMonoPS-Regular-fontforge.otf'
  #    - Click 'Generate'.
  # 3. Quit FontForge without saving.
  #
  # Move new <GSUB> into original font.
  afdko sfntedit -x GSUB=temp.GSUB NimbusMonoPS-Regular-fontforge.otf
  afdko sfntedit -a GSUB=temp.GSUB NimbusMonoPS-Regular.otf
  rm temp.GSUB
  #
  # Verify edits.
  ttx NimbusMonoPS-Regular-fontforge.otf
  meld NimbusMonoPS-Regular-org.ttx NimbusMonoPS-Regular-fontforge.ttx
  # ==> FontForge renumbered and rearranged most glyphs.
  # ==> FontForge created empty or useless entries <FFTM>, <GDEF> and <GPOS>.
  # ==> FontForge optimizes entries in <GPOS> by unifying identical entries,
  #     shortening <FeatureList> from 96 to 8 entries.
  # ==> Generally, everything looks fine.
  # ==> Optimized font is a nice 1508 bytes smaller.
  #
  # Verify results.
  ttx NimbusMonoPS-Regular.otf
  meld NimbusMonoPS-Regular-org.ttx NimbusMonoPS-Regular.ttx
  #
  # Verify tables.
  afdko spot -F NimbusMonoPS-Regular-org.otf > bla1
  afdko spot -F NimbusMonoPS-Regular.otf > bla2
  meld bla1 bla2
  rm bla1 bla2
  # ==> Entries 'liga' are gone, many others just renumbered.

Voilà, all ligatures for all scripts removed, font working better than ever.

Nicely, with font Nimbus Mono, table GSUB is identical for all font variants.
This allows to simply copy this table from Regular to the others.

  afdko sfntedit -x GSUB=temp.GSUB NimbusMonoPS-Regular.otf
  afdko sfntedit -a GSUB=temp.GSUB NimbusMonoPS-Bold.otf
  afdko sfntedit -a GSUB=temp.GSUB NimbusMonoPS-Italic.otf
  afdko sfntedit -a GSUB=temp.GSUB NimbusMonoPS-BoldItalic.otf
  rm temp.GSUB
ghost commented 2 years ago

BTW., if you really want to remove all substitutions, it's as simple as (untested):

afdko sfntedit -d GSUB <font>.otf

P.S.: I won't do this, because I prefer to change only things where I can prove misbehavior and also test the result.

fabiangreffrath commented 2 years ago

@chris-liddell @deekej This issue might have a bit higher priority than many others, as it directly affects metric compatibility of the fonts. Could you please consider taking this upstream to (URW)++ again or, given lack of response, consider applying the patch that @merchantsedition provided - effectively forking the fonts? Thank you!

deekej commented 1 year ago

Hello @fabiangreffrath - unfortunately I no longer work as a package maintainer in Red Hat, and in my spare time I don't have capacity to incorporate this into Fedora either. I suggest contacting the current maintainer for urw-base35-fonts in Fedora if you wish to see these changes there... ;)

fabiangreffrath commented 1 year ago

I suggest contacting the current maintainer for urw-base35-fonts in Fedora if you wish to see these changes there... ;)

It's not that I want to see the patch applied in distribution packages, else I would have already applied it to the Debian package. I want it to be applied upstream and then migrate to distributions the natural way.

chris-liddell commented 1 year ago

Sorry for the lengthy lack of response, we've been trying to get some kind of communication "upstream" and have basically had no luck. So, when I have some time available, I will look at incorporating the patch @merchantsedition supplied, and some other changes.

I have, frankly, been extremely reluctant to do this, because the previous time the URW++ fonts were forked, it caused us no end of headaches, but I think we're out of options :-(

n8willis commented 1 year ago

TTF and T1 formats don't have such feature sets, querying these variants with afdko spot -F comes up just empty.

The OTF format has more features than just ligatures. There's also a mechanism to replace 1/2 with ½ (and many similar) somehow. These are not ligatures, so far I have no idea how to take advantage of these features, or whether they should be removed from a monospaced font as well. Unlike with ligatures, typing 1/2 in a generic text editor doesn't replace anything.

I don't mean to jump in late to a discussion here, but I'm noticing several indicators that there's some confusion of terms in the thread, which has the potential to cause unwanted ill effects for eventual users downstream.

liga is a registered tag name for an OpenType Layout GSUB substitution feature. It's defined as "recommended" to be on by default, but can be disabled by user-level applications or settings (or markup, such as CSS). As at https://learn.microsoft.com/en-us/typography/opentype/spec/features_ko#tag-liga

Whether a particular liga substitution replaces two glyphs with one, or two glyphs with two other glyphs, or some other permutation, is not knowable in advance just from the feature name. Nor is it knowable in advance whether the ligated glyph(s) will have the same overall width or different width from the pre-substitution. In cursive scripts (say, Arabic, for example), cutting out liga by feature name would break functionality (like the lam-alef ligature, which is mandatory). Similarly, italic fonts in Latin can and do have ligature substitutions for important reasons, such as preventing collisions between glyphs. In a lot of languages, the substitutions for base-and-mark combinations use these mechanisms to make things readable.

So the blanket notion that "monospaced fonts shouldn't have ligatures" is not true. The slightly more focused "monospaced fonts should not have GSUB substitutions active by default that change the width of the sequence" is maybe a little more defensible, but then again, reading the thread it sounds like that is purely a matter of taste, I suspect coming from folks who consider monospace fonts as being exclusively used for terminals or code editing.

That's not the only use case. That's what the frac feature (which does the fraction substitutions mentioned above) illustrates. It's a ligature substitution, which enables people to use those ½ style fractions in their documents, even though most keyboards don't have a key for them. It's also meant to be enabled/disabled by user-level application code, such as the text style menus or CSS settings. And you genuinely do not have guarantees that liga or frac or any other substitution will alter

In both cases, it's extremely dangerous to chop out GSUB features and it can break people's documents. Requesting the change be made to an "off my default" GSUB lookup by the upstream font publisher is half of the solution.

The other half is that code-listing widgets and terminal apps should have optional GSUB features like liga disabled at the app/toolkit layer if people want their text to look that way. That needs issues being opened on those projects. This was an issue a few years ago with ligatures in LibreOffice presentations (IIRC) when letter-tracking was increased: before people raised the issue, "a f f i n e" with some added space would get rendered as "a ffi n e". The application code needed fixing, not the font. That also needs to be done here, if people in coding/terminal situations are seeing width-changing ligatures where those should not be possible.

There's some decent technical rabbit-hole reading to be found at https://simoncozens.github.io/fonts-and-layout/features.html that people might find helpful.

Also, I know I've already gone super long and doubt anyone's still reading, but GSUB and GPOS are part of Open Type Layout, which is an extension, and both GSUB and GPOS are 100% valid tables in a file with a .ttf extension. OpenType as a whole is a superset of TrueType (meaning every TrueType font is also an OpenType font; they're not mutually exclusive), and the filename extension doesn't tell you which tables might be inside, especially with OpenType Layout and the other extensions to the original published spec.

tomty89 commented 1 year ago

but then again, reading the thread it sounds like that is purely a matter of taste, I suspect coming from folks who consider monospace fonts as being exclusively used for terminals or code editing.

Not so sure about this. I mean, I can't rule out that a fraction of people in the world might be into appearance like " a ffi n i t y", but I'm also quite sure most sane people don't have such a "taste" (more like a fetish, if I may).

Or maybe what you mean is that the ligated ffi should have the width of three characters instead of one? Otherwise it's a renderer's fault?

Anyway, this really isn't a "background" or "context" thing. It's just the same when it comes to website or PDF. (Actually, I wouldn't have even participated in this discussion if it's a terminal or code editor thing. Does anyone even use nimbus mono for that anyway?)

And the examples you have given (Arabic, italic latin) don't seem really persuasive, as you just seem to assume that what's appropriate for them should apply in monospace font as well. (I mean, is monospace even a thing when it comes to Arabic?)

Nor is it knowable in advance whether the ligated glyph(s) will have the same overall width or different width from the pre-substitution.

In my opinion, the fact that it's "not knowable" (or, "can't be guaranteed") is itself quite a strong reason that monospace font should not support that at all. Otherwise, what's the point / definition of "monospace"? (Which is why I feel like, yeah maybe there's a way for a program to find out whether a font is of the monospace family and determine whether ligature is enabled based on that, but it doesn't sound like its responsibility either.)

(I can't help but think of the "origin" of monospace font. Isn't that typewriter? And is ligature a thing in that? If so, perhaps it tells us how ligatures should look like / be done in monospace font?)

meaning every TrueType font is also an OpenType font

I'd say only the other way round is true. But meh. It's not really relevant to the discussion, is it?

P.S. I am not replying to like make sure this "downstream patching" gets in at the end. To be frank I don't really like the idea, as it does smells like "something not gonna end well". Neither am I trying to stop it from happening / intervene though.

n8willis commented 1 year ago

can't rule out that a fraction of people in the world might

They still can use the ligature if they want to, because application settings and markup permit them to activate it. Breaking the font for them is what takes that flexibility away.

I'm also quite sure most sane people don't have such a "taste" (more like a fetish, if I may).

This is pejorative straw-man stuff and not germane to a serious discussion.

Anyway, this really isn't a "background" or "context" thing. It's just the same when it comes to website or PDF.

No, it's 100% a context thing. Browser or DTP app included. Did the original user see the "fi" ligature they don't want inside a <tt> or <code> element in Firefox? Then it's a Firefox bug. Did they see it in an inline-code-block in their Markdown editor? Then it's a Markdown editor bug. Did they see it in vim? Then it's a vim bug. If you don't know where they encountered it, then you don't know where the bug lies.

That's why breaking the font for all users isn't the solution. The purpose of GSUB's suite of lookup tags is that applications can set the combination of feature settings appropriately for individual elements. E.g., tabular numerals ON by defaults in spreadsheets, but OFF by default in text-paragraphs.

And the examples you have given (Arabic, italic latin) don't seem

I am talking exclusively about monospaced fonts.

(I mean, is monospace even a thing when it comes to Arabic?)

Yes. If you are genuinely needing to ask that, then that suggests you are not yet sufficiently brushed up on the background of how and where OpenType features are used in fonts and why. You really would do well to read the specification. Even the intro to OpenType Layout would probably make a lot of things click that are jumbled elsewhere in bug reports: https://learn.microsoft.com/en-us/typography/opentype/spec/ttochap1

Just as monospaced italics are a thing. Monospace italics and monospace bolds & other weights are how code-highlighter engines distinguish different semantics.

Additionally, ligature-substitution lookups are how all those "coding ligature" fonts popularized in the past few years function. I don't find many of those substitutions useful, and some of them (like double-wide greater-than-or-equal) I find downright unappealing. But they can be optionally activated or deactivated because of the GSUB feature-tag mechanism. Which again is why trying to hack them out of the font binary in the upstream package is not the solution; user X might not like the ligatures, but breaking the font for all users causes harm to them.

I'd say only the other way round is true.

Well, it's not a matter of opinion, it's a specification, and you have it 180 degrees backwards in that statement.

More importantly, here:

It's not really relevant to the discussion, is it?

It is relevant. I brought it up because of this in comment 1265993694:

This applies to OTF fonts only, as T1 and TTF formats don't support ligatures at all, so there can't be something to remove.

which is incorrect. Even outside of GSUB and GPOS (which .ttf files can — and do — include), .ttf fonts can include ligatures in other tables. Most often the mort and morx tables, which are not part of OpenType Layout.

I don't really like the idea, as it does smells like "something not gonna end well".

On that we fully agree. :)

n8willis commented 1 year ago
  • Use FontForge to remove these ligature entries.
  • Use Adobe Font Development Kit for OpenType (AFDKO) tool 'sfntedit' to move table 'GSUB' (which contains these ligature definitions) from the FontForge edited version to the original version.
  • Use Fonttools' 'ttx' and AFDKO's 'spot' to verify all this.

On a different note, FontForge can't really be relied upon to do surgery like this on a binary font file. GSUB and GPOS lookups can chain and "reverse chain" and be defined with context referring to other lookups, and FontForge doesn't handle that. Removal could break those interdependencies and FontForge doesn't attempt to do constraint-satisfaction stuff like would be required to close all the loops.

I wouldn't particularly rely on AFDKO for that either, but mostly because it's less used and might not have been tested.

hvdijk commented 1 year ago

No, it's 100% a context thing. Browser or DTP app included. Did the original user see the "fi" ligature they don't want inside a <tt> or <code> element in Firefox?

I think it was a <pre> element, but it's been about a year and a half since I reported this so I could well be misremembering. But you're mischaracterising my report. I don't care whether I see a ligature, I care that the layout gets all messed up as a result. A font that is claimed to be metrically compatible with Courier needs to have the same metrics as Courier regardless of whether the applications enable or disable ligatures, because the font substitution will be applied regardless of whether ligatures are enabled, and right now, when ligatures are enabled, they aren't, Nimbus Mono-with-ligatures has different metrics than Courier-with-ligatures. If Nimbus Mono were to have an fi ligature that has the same width as an f character followed by an i character, that would be just fine.

n8willis commented 1 year ago

But you're mischaracterising my report. I don't care whether I see a ligature, I care that the layout gets all messed up as a result.

I get that completely. I'm saying that the problem you're describing is one of behavior, not of metrics.

The metrics are defined as widths & advances of each glyph, in the hmtx table (for horizontal text). If the two fonts have the same values for the same glyphs/codepoints, then they have the same metrics. GSUB lookups are not altering that. What you're encountering is not about the metrics, it's about other stuff.

If Nimbus Mono were to have an fi ligature that has the same width as an f character followed by an i character, that would be just fine.

If Courier also had an fi ligated character that had that same width, then the two fonts would still be metrically compatible. The difference in the behavior is about whether or not this substitution happens when you use it, not about the widths and advances.

If Nimbus Mono and Courier have fi ligated characters that are different widths, then the two fonts have different metrics, regardless of whether or not each one performs the "f, i"->"fi" substitution always/sometimes/never. The test case there would be if somebody had a document where they intentionally put in the "fi" ligature by its Unicode codepoint (U+FB01): if the two fonts have different widths for that, they're not metrically compatible, and that person would see it because their document (or screen or whatnot) would have its text reflow.

That's the difference between metric compatibility and behavior issues.

tomty89 commented 1 year ago

Did the original user see the "fi" ligature they don't want inside a \<tt> or \<code> element in Firefox? Then it's a Firefox bug.

What I really mean by background / context does not matter is, I think when it comes to monospace, it's not about whether "some of us are used to how it should look like", or whether a font format spec states that something is supported or not, or whether the font is used for certain "semantic" reason. Rather it's a matter of the definition of monospace.

It's quite a intuitive / simple math thing as far as I can see -- regardless of whether ligature should be done / is desired, the first rule is that the after ligation "it" should be still the same width as others, with "it" ~either~ being ~the whole ligated unit or~ (I was really just thinking of ½ for this) each character in it. If for whatever reason (including but not limited to the font format spec I'm indeed not familiar with), then I'd say ligature should not be allowed at all.

With that said I'm not ruling out it's not something to do with the font, but perhaps yes, everyone just screwed up historically in monospace rendering. But at the same time, IMHO whether we can / should just "cut off" ligature in Nimbus Mono can still depend on characters it has / would ever cover, even if the "dangerous / wrong" case you mentioned are real.

If Nimbus Mono were to have an fi ligature that has the same width as an f character followed by an i character, that would be just fine.

And this is probably the most important point I would like to emphasize as well. Personally speaking I don't even care / know much about metric compatibility. The real problem is that if ligated ffi is not of the same width as not-ligated ffi, then something (renderer or the font) must be wrong. (Although to be frank, I'd probably find it odd even if the width is the same, but that I can sort of accept it to be "a matter of taste". For now. I'm still thinking about the typewriter / origin matter.)

It is relevant. I brought it up because of this in https://github.com/ArtifexSoftware/urw-base35-fonts/issues/31#issuecomment-1265993694:

I think that's why "the other way round" thing matters. As you said, OpenType is superset of TrueType, which means there's still a set for TrueType. And regardless of whether that has something to do with the implementation of font renderer or so, or whether the ttf available here really don't have the ligature table whatsoever, de facto speaking, the issue is apparently not observed in the ttf here.

I think what the comment really means is, "if we can survive with the practically no-ligature ttf files here, we would be able to survive with such otf as well. (I don't think its point is about "right or wrong". Btw, AFAICT it's undeniable that the ttf here are not just the otf with different extension name.)

hvdijk commented 1 year ago

But you're mischaracterising my report. I don't care whether I see a ligature, I care that the layout gets all messed up as a result.

I get that completely. I'm saying that the problem you're describing is one of behavior, not of metrics.

I'm not too interested in a semantics game, but fine, let's go. A good starting point is always Wikipedia, and following its sources. It says:

Typefaces that can be substituted for one another in a document without changing the document's text flow are said to be "metrically identical" (or "metrically compatible").

Its sources support this definition:

They are metrically compatible with the original Windows core set so documents created with either core set of fonts will not reflow.

The CALS profile considers any rendering of a requested font conforming if the rendering is "metrically identical" to the metrics of the requested font. This means that the placement and alignment of the string and the placement, size, and shape of individual characters (Le., the drawn portions of the character cells) are measurable identical.

This means that if you import a document using Calibri into LibreOffice and you don’t have Calibri or a metrically compatible font installed, your document layout would change as the font LibreOffice would need to use instead have letters that might in general be slightly wider for instance.

It is not just about the metrics of individual glyphs. If different sets of ligatures causes text to reflow when switching between fonts, the fonts are, by this definition, not metrically compatible. And that's exactly what's currently happening with Courier and Nimbus Mono, because Nimbus Mono uses ligatures where Courier does not, and Nimbus Mono's ligatures are not of the same size as Courier's non-ligature characters.

This is the definition of metric compatibility I am using. If you object to this definition and want to use a different term that means the same thing, I have no issue with that, but I have not seen an alternative single term that covers all this and only this in your message, and given that others are using metric compatibility exactly the same way as I am, I see no issue with continuing to use it the way I am.

hvdijk commented 1 year ago

Personally speaking I don't even care / know much about metric compatibility.

For a bit more context, someone had created a preformatted plain text table using Courier, like so:

first      sixth
second     seventh
third      eighth
fourth     ninth
fifth      tenth

This rendered badly when Nimbus Mono was installed, because first turned to first, and fifth to fifth, and the second column became misaligned as a result. It wasn't a subtle difference that was only visible to the trained eye, it was painfully obvious.

fabiangreffrath commented 1 year ago

Ths is not about personal taste or "getting used to ligatures".

Your app enables ligatures -> you render text with Courier -> you won't see ligatures Your app enables ligatures -> you render text with Nimbus Mono -> you will see ligatures ==> metric compatibility broken

It's as simple as that.

chris-liddell commented 1 year ago

To throw another spanner into the works here, the claim about "metric compatibility" really only applies to the Type 1 fonts and their compatibility with the Postscript Level 2 font set, particularly being used in a Postscript/PDF context.

And since the way both Postscript and PDF can sort of use OpenType fonts means GSUB and other "advanced" features don't apply, that compatibility remains in that context.

@n8willis does raise a valid point, too, that these fonts have been out there and being used in their current state. Chopping them around could well cause inconvenience to users who have documents that rely on their current form.

chris-morgan commented 1 year ago

I shall be blunt since there’s so much prevarication and attempted justification going on: the current behaviour is blindingly obviously a bug and undesirable in absolutely every case, no exceptions, and no one in their right mind will be depending on the current form or do anything other than breathe a sigh of relief if it’s fixed—quite apart from it being difficult to truly depend on the current behaviour.

(But I should clarify that I’m speaking of Nimbus Mono here. I don’t venture anything with respect to other fonts, and they weren’t the subject of the initial report; if there’s any discussion to be had on any other fonts, separate Nimbus Mono from the rest.)

ewtoombs commented 11 months ago

The archlinux wiki describes a workaround for this bug. Also, I agree with @chris-morgan, and will add that on top of looking like total dogshit, it misaligns any code or ASCII art that assumes "ffi" occupies three spaces. Fix it!

# Static information about the filesystems.
# See fstab(5) for details.

# <file system>          <dir>       <type> <options>             <dump> <pass>
/dev/mapper/office-root  /           ext4   rw,relatime,discard   0      1
/dev/mapper/home         /home       ext4   rw,relatime,discard   0      2
patrakov commented 11 months ago

The workaround does not work for me. Thus, I created a PKGBUILD that packages the TTF variant and let it replace the original package. And I recommend all distributions do that until the OTF version is fixed.

# Based on the original gsfonts PKGBUILD, which has the following headers:
# Maintainer: David Runge <dvzrv@archlinux.org>
# Contributor: Gaetan Bisson <bisson@archlinux.org>
# Contributor: Firmicus <francois.archlinux.org>
# Contributor: dorphell <dorphell@archlinux.org>

_name=urw-base35-fonts
pkgname=gsfonts-ttf
pkgver=20200910
pkgrel=3
pkgdesc="(URW)++ base 35 font set (ttf format)"
url="https://github.com/ArtifexSoftware/urw-base35-fonts"
conflicts=(gsfonts)
provides=(gsfonts)
arch=(any)
license=(AGPL3)
source=($url/archive/$pkgver/$_name-$pkgver.tar.gz)
sha512sums=('71fb27baadf5abc4ff624cdede02038681acd5fffdc728a5b2e7808713b80cb2f2174f90a1862e69d390c4434c49d5167ab095100032fa3ba80b586eb8ae51d1')
b2sums=('77305946e6ab6e576cca7b62da731709bf4acaeb7181f89aef032f922acfaad59735a651d73ff8793c2cf32a23f45efaf5322568892023d4eae9b2278f681890')

package() {
  local _config _config_path
  cd $_name-$pkgver

  install -vDm 644 fonts/*.ttf -t "$pkgdir/usr/share/fonts/$pkgname/"
  install -vDm 644 appstream/*.xml -t "$pkgdir/usr/share/metainfo"

  install -vdm 755 "$pkgdir/usr/share/fontconfig/conf.default/"
  for _config in fontconfig/*.conf; do
    _config_path="$pkgdir/usr/share/fontconfig/conf.avail/69-${_config##*/}"
    install -vDm 644 "$_config" "$_config_path"
    ln -srt "$pkgdir/usr/share/fontconfig/conf.default/" "$_config_path"
  done
}
hvdijk commented 11 months ago

A fix was already provided in the comments by @merchantsedition in https://github.com/ArtifexSoftware/urw-base35-fonts/issues/31#issuecomment-1261458443. It still hasn't been applied, but for people looking to fix this locally on their systems, I would suggest just going with that.