Crash without error when patching custom Iosevka .ttc

[Bahnschrift]

🗹 Requirements

• [X] I have searched the issues for my issue and found nothing related and/or helpful
• [X] I have searched the FAQ for help
• [X] I have searched the Wiki for help

🎯 Subject of the issue

Experienced behavior: When trying to patch a custom build of Iosevka with the complete set of icons, the patcher eventually crashes with no error message. When patching on windows, it crashes when patching the 6th font in the .ttc. However, on WSL, it crashes when patching the 51st font (out of 54). The crash does not occur if I use the --careful flag, however this is not a desirable workaround.

🔧 Steps to reproduce

The Iosevka build script I am using is as follows:

[buildPlans.iosevka-custom]
family = "Iosevka Custom"
spacing = "normal"
serifs = "slab"
export-glyph-names = false

[buildPlans.iosevka-custom.ligations]
enables = [
    "centre-ops",
    "arrow",
    "arrow2",
    "trig",
    "eqeq",
    "exeq",
    "ineq",
    "eqeqeq",
    "exeqeq",
    "exeqeqeq",
    "eqexeq",
    "ltgt-diamond",
    "ltgt-slash-tag",
    "brst",
    "plusplus",
    "kern-dotty",
    "kern-bars",
    "logic",
    "llgg",
    "llggeq",
    # "dot-as-operator",
    "bar-triggers-op-centering",
    "html-comment",
    "brace-bar",
    "brack-bar",
    "connected-underscore",
    "connected-number-sign",
    "connected-tilde-as-wave",
    "connected-hyphen-as-semi-dashed-line"
]

[buildPlans.iosevka-custom.variants]
inherits = "ss20"

[buildPlans.iosevka-custom.variants.design]
zero = "dotted"
asterisk = "penta-low"
caret = "high"
paren = "flat-arc"
ampersand = "lower-open"
at = "fourfold-solid-inner"
lig-ltgteq = "slanted"
question = "corner"

[collectPlans.iosevka-custom]
from = ["iosevka-custom"]

I've uploaded the .ttc for this font here.

On windows, I am patching the font with fontforge.exe -lang py --script font-patcher iosevka-custom.ttc -c --mono -w, and on WSL I am patching with ./font-patcher iosevka-custom.ttc -c --mono.

[Finii]

I guess this is a out-of-memory situation.

Iosevka is rather big, and then holding that 50 times in memory, that is just not possible.

I would recommend patching one font after another and then combining all fonts into one ttc.

[Bahnschrift]

That could be the case, but I'm not entirely convinced. Unless something to do fontforge having some custom memory limit, I should have enough system memory to hold all the fonts (32GB), and on windows at least the crash occurs when fontforge is still using less than 2GB.

If this is the problem, maybe it would be possible for me to modify the patcher script so that each font is saved and closed after patching, then give all the filenames as arguments to fontforge.generateTtc? I'll give that a try a bit later. If that doesn't work, I'll follow your suggestion patching them all individually.

Is there any way I could get the patcher / fontforge to produce more logs and information about what's happening?

[Finii]

Hmm,

lets try...

after some time

Being at font 15 the size is at ~5G, so maybe 18G at the end, lets see... (tumble thumbs)

27 -> 10G:

35 -> 12.4G

42 -> 15G

47 -> 16.7G

50 -> 17.8G

53:

54

54 finished

Now it's exporting ...

(uups I still have debug code in my fontforge...)

[Finii]

Export finished, now it's tweaking:

[Finii]

Tweaking will probably take a long time, but the result-font-file has already been created.

(Tweaking is a post-run script that manipulates the font file directly without fontforge (because that can not do some stuff).

[Finii]

Tweaking progresses slowly, memory constant.

It does in fact binary patch the file on disc to correct some info in the generated font file directly, here the lowestppem; fontforge does not support to even read that, let alone write.

I used a rather recent (self built) fontforge:

Maybe older (released) fontforge versions have a memory problem, or your system resources are not big enough? ulimits? This was the PR I have been working on: https://github.com/fontforge/fontforge/pull/5136 (thus the debug code)

[Bahnschrift]

Interesting, so it seems to be working fine for you? It seems a bit odd that is behaves differently on windows and WSL, and I'm guessing you're running native linux?

I also tried patching the same font but without the serifs (commented out serifs = "slab" in the build plans), resulting in a slightly smaller .ttc (6MB less), and it worked fine on WSL.

I'm using the latest release of fontforge (20220308), but it looks like the memory on WSL is capped at 16GB + 4GB swapfile, so I'll try increasing that and see if makes a difference.

[Finii]

windows [...] WSL

I must admit ... suggestion: Use proper linux ;-)

Is there any way I could get the patcher / fontforge to produce more logs and information about what's happening?

Not really. The script does not prevent any fontforge output. I guess it crashed within a fontforge call and so you see nothing in the script output.

If this is the problem, maybe it would be possible for me to modify the patcher script so that each font is saved and closed after patching, then give all the filenames as arguments to fontforge.generateTtc? I'll give that a try a bit later. If that doesn't work, I'll follow your suggestion patching them all individually.

Well, fontforge needs to hold all fonts in memory before a ttc generate can be initiated. Saving the individual font files ... and then opening all files will not help here. You could patch all files individually and use some other tool to combine them into one ttc.

Just looked at the fonttools and they also need to hold all ttf fonts in memory before the ttc can be generated. Probably there is no memory-friendly 'just copy from many input files to one destination file' tool.

[Finii]

I guess it's working here, yes. Well, the patching worked, the tweaking (which is just one raw file byte manipulation script I wrote) is till in progress because it is so slow :grimacing:

The March 2022 Release is definitively good, older revisions could not even patch Iosevka at all.

If you just need this one patch run, I can give you my ttc; but that will not help you if you expect to do it again in the future.

Maybe it is also worth to try the Nerd Font docker container? I have no clue how Windows handles memory resources of those.

A yes, I'm on Ubuntu 22.10 here, just my small laptop 'log into the company's servers via NX' frontend thing, not very powerful.

[Finii]

give all the filenames as arguments to fontforge.generateTtc?

It does not take filenames, it does only take font objects (others, i.e. opened fonts):

[Bahnschrift]

I just tried with more memory allocated to WSL, and it works fine. Not sure what the issue was on windows, but at least for WSL memory was definitely the issue.

[Bahnschrift]

Now I'm getting a problem with ligatures not working in IDEs (but still working in windows terminal?) on the patched fonts. Guess I'll take a look around and see if I can see if I can find anything related.

[Finii]

Now I'm getting a problem with ligatures not working in IDEs (but still working in windows terminal?)

Not working at all, or misbehaving? It should not be a difference if you use some IDE or a barebone-IDE called windows terminal. Often you can disable ligatures and/or some might not support ligatures at all. The ligatures do work with the unpatched fonts in that specific IDEs?

[Bahnschrift]

The ligatures work fine with the unpatched fonts in all scenarios that I would expect ligatures to work. They also work with the patched fonts in my terminal emulator - Windows Terminal. They don't however work in either IntelliJ IDEA or VSCode, even with all the appropriate ligature settings enabled.

I'll do some more poking around and try and work it out. Maybe I should open another issue?

[Finii]

Maybe I should open another issue?

That's always best :-) Thank you

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity (i.e. last half year) after it was closed. It helps our maintainers focus on the active issues. If you have found a problem that seems similar, please open a new issue, complete the issue template with all the details necessary to reproduce, and mention this issue as reference.