improving generation of large fonts

[u-fischer]

Some fonts (CJK fonts mostly) are so large that the generation of the lua file takes quite long or even fails eating up the memory (I had to use 64bit luatex for some of them).

Is the code suggested in https://tex.stackexchange.com/a/479753/2388 a possible option and can there be some interface for it?

fonts.handlers.otf.readers.registerextender {
    name   = "remove names from Noto Sans CJK SC",
    action = function(fontdata)
        if string.match(fontdata.metadata.fullname, "Noto Sans CJK SC") then
            texio.write_nl("Trimming font " .. fontdata.metadata.fullname)
            for k, v in next, fontdata.descriptions do
                v.name = nil
            end
        end
    end
}

[Firestar-Reimu]

I am thinking of rewriting luaotfload in C and only use lua to make interfaces

That solution does not always work. See this:

\documentclass[UTF8, fontset=none]{ctexart}
\setCJKmainfont{Noto Sans CJK SC} % Noto CJK
% \setCJKmainfont{Adobe Song Std} % Adobe Song
% \usepackage{luacode} % with luacode
% \begin{luacode*}
% fonts.handlers.otf.readers.registerextender {
%     name   = "remove names from Noto Sans CJK SC",
%     action = function(fontdata)
%         if string.match(fontdata.metadata.fullname, "Noto Sans CJK SC") then
%             texio.write_nl("Trimming font " .. fontdata.metadata.fullname)
%             for k, v in next, fontdata.descriptions do
%                 v.name = nil
%             end
%         end
%     end
% }
\usepackage{zhlipsum}
\begin{document}
\zhlipsum[-]
\zhlipsum[-][name=zhufu]
\end{document}

this is a 30-page all-Chinese article

I tested with this:

import time
import os

a = time.time_ns()
os.system("xelatex sample1.tex > /dev/null")
b = time.time_ns()
print("Noto CJK - xelatex", b-a)

a = time.time_ns()
os.system("lualatex sample1.tex > /dev/null")
b = time.time_ns()
print("Noto CJK - no luacode", b-a)

a = time.time_ns()
os.system("lualatex sample2.tex > /dev/null")
b = time.time_ns()
print("Adobe Song - no luacode", b-a)

a = time.time_ns()
os.system("lualatex sample3.tex > /dev/null")
b = time.time_ns()
print("Noto CJK - with luacode", b-a)

The result is:

python test.py 
Noto CJK - xelatex 715470122
Noto CJK - no luacode 1676104479
Adobe Song - no luacode 1235540826
Noto CJK - with luacode 1716780934

[zauguin]

I'm not sure what you are saying. This issue is about some versions having very slow cache generation for huge fonts while your experiments are all running with preexisting caches. Therefore it would be very surprising if adding that codeblock would make any difference.

Beside that @u-fischer I'm relatively sure that in current versions these names are stripped anyway, making this block rather useless. Could you test whether adding this still makes any difference to generation times on Windows?

[u-fischer]

@zauguin No there is no time difference. The cjk fonts works fine now, also without a 64bit luatex. One could simply close this here imho.

[Firestar-Reimu]

So the current problem is that it is slow after generating font caches? This test also found this.

Lua­TeX 在 TikZ 和 zhlipsum 测试中的表现都比较好，大概是利用 Lua 进行了优化；但在 Source Han 测试中调用字体花费了过多的时间，这还是在之前已经做好字体缓存的前提下进行的测试 LuaTeX performs better in both TikZ and zhlipsum tests, probably optimized with Lua; but it takes too much time to call fonts in Source Han test, which is done on the premise that font caching has been done before test

[zauguin]

@Firestar-Reimu If current versions are very slow for some fonts please open an issue for that, but in my experiments the loading time seems reasonable. (My system currently uses 0.07s to load Noto Sans CJK SC, half of which is the time needed by LuaTeX itself to parse the font table we send from Lua)

[Firestar-Reimu]

@zauguin See #279 , thanks