Building pdf-docs with texlive-2023

[strogdon]

First, I have not been able to build the above with vanilla. The stopper appears to be

Writing index file references.idx
(./references.aux) (/usr/share/texmf-dist/tex/latex/base/ts1cmr.fd)
(/usr/share/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
) (/usr/share/texmf-dist/tex/latex/epstopdf-pkg/epstopdf-base.sty
(/usr/share/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg))
*geometry* driver: auto-detecting
*geometry* detected driver: luatex
(./references.out) (./references.out)<<oml.cmap>><<oms.cmap>><<omx.cmap>>
(/usr/share/texmf-dist/tex/latex/amsfonts/umsa.fd)
(/usr/share/texmf-dist/tex/latex/amsfonts/umsb.fd)
(/usr/share/texmf-dist/tex/latex/jknapltx/ursfs.fd)<<ot1.cmap>> [1{/var/lib/tex
mf/fonts/map/pdftex/updmap/pdftex.map}] [2] (./references.toc
! Undefined control sequence.
l.1 \babel@toc
             {english}{}
?
! Emergency stop.
l.1

which suggest something is amiss with texlive. However, when building the docs for s-o-g I'm unable to get past the html-docs with failure

Error building the documentation.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/lib/python3.11/site-packages/sage_docbuild/__main__.py", line 508, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/sage_docbuild/__main__.py", line 504, in main
    builder()
  File "/usr/lib/python3.11/site-packages/sage_docbuild/builders.py", line 824, in _wrapper
    getattr(DocBuilder, build_type)(self, *args, **kwds)
  File "/usr/lib/python3.11/site-packages/sage_docbuild/builders.py", line 162, in f
    runsphinx()
  File "/usr/lib/python3.11/site-packages/sage_docbuild/sphinxbuild.py", line 327, in runsphinx
    sys.stderr.raise_errors()
  File "/usr/lib/python3.11/site-packages/sage_docbuild/sphinxbuild.py", line 263, in raise_errors
    raise OSError(self._error)
OSError: WARNING: autodoc: failed to import module 'arith_sage' from module 'sa
ge.libs.flint'; the following exception was raised:

I'm unable to find arith_sage.

[kiwifb]

That's not ones I have seen. For the first one, it would look like a missing language pack. The second one is of interest because I believe that means you are on the vbraun branch. That missing file should have made its appearance in a merge a few hours ago that I have not tested yet.

[strogdon]

I pulled from master

/var/db/repos/sage-on-gentoo # git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

I will check language packs. I was missing a few. I had to emerge texlive directly.

[kiwifb]

I have changed how I deal with the vbraun branch slightly. If you have installed the files from the vbraun.settings folder, you are still pulling the vbraun branch even if the overlay has not been switched over. The overlay switch over means that you may have different patches, because it has just been merged for 10.3.beta4 the patches are identical. If you have build logs, it should tell you which git repo you are pulling sage from at the beginning of the log.

[strogdon]

So yes I had pulled from the vbraun branch. However, I created no vbraun symlinks. Is this working correctly? Must I remove the vbraun.settings folder from the overlay?

[strogdon]

I removed the vbraun.settings folder and I'm still pulling from https://github.com/vbraun/sage.git when building sage-doc. In the sage-doc ebuild there is EGIT_REPO_URI="https://github.com/vbraun/sage.git". Is this correct?

[kiwifb]

No it is not. It looks like a mistake made it to the sage-doc ebuild on the master branch. The only ebuild that currently does not use the sage-git eclass. I should have paid more attention. I will review the vbraun settings and the ebuild shortly.

[kiwifb]

OK, fixed the repo source in the ebuild, it may take up to 15 mn for the gentoo mirror to synchronise.

[strogdon]

I got through the html-docs but I have a SANDBOX VIOLATION when building the pdf-docs which seems to originate with

LaTePackage fancyhdr Warning: \headheight is too small (12.0pt): 
X Warn(fancyhdr)                Make it at least 15.0pt, for example:
ing: H(fancyhdr)                \setlength{\headheight}{15.0pt}.
yper ref(fancyhdr)                You might also make \topmargin smaller to compensate:
e
rence (fancyhdr)                \addtolength{\topmargin}{-3.0pt}.
`
functional_programming:functional-programming' 
on page 346 undefined on input line 26844.

][73] * ACCESS DENIED:  mkdir:              /var/cache/fonts/pk/modeless/public
mkdir: cannot create directory ‘././var/cache/fonts/pk/modeless/public’: Permission denied
mktexpk: /usr/share/texmf-dist/web2c/mktexdir /var/cache/fonts/pk/modeless/public/tex-gyre failed.

Package fancyhdr Warning: \headheight is too small (12.0pt): 
(fancyhdr)                Make it at least 15.0pt, for example:
(fancyhdr)                \setlength{\headheight}{15.0pt}.
(fancyhdr)                You might also make \topmargin smaller to compensate:

(fancyhdr)                \addtolength{\topmargin}{-3.0pt}.

[74] [163] [76kpathsea: Appending font creation commands to missfont.log.
dvipdfmx:warning: Could not locate a virtual/physical font for TFM "ec-qhvb".
dvipdfmx:warning: >> There are no valid font mapping entry for this font.
dvipdfmx:warning: >> Font file name "ec-qhvb" was assumed but failed to locate that font.
dvipdfmx:fatal: Cannot proceed without .vf or "physical" font for PDF output...

No output PDF file written.
]
Глава 6.
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  dvipdf: Command for 'dvipdf' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.
make[2]: *** [Makefile:30: a_tour_of_sage.pdf] Error 12
make[2]: Leaving directory '/var/tmp/portage/sci-mathematics/sage-doc-9999/work/build_doc/latex/ja/a_tour_of_sage'
Error building the documentation.

Have you had any issues? The folder /var/cache/fonts/pk/modeless/ exists but the public folder cannot be created. I am the owner of the modeless folder, not root.

[strogdon]

This may not be the issue but dev-libs/kpathsea has not been updated to the tex-overlay version.

[kiwifb]

That actually could be it. I did not test any other language than English and French. Did you install freefont (not to be mistaken with freefonts)?

[kiwifb]

I should note that I had to already add instructions in the ebuild for a sandbox violation where luatex check whether it can write on the system as part of a test looking for a location it can write to, and it is caught by the sandbox. So, I had to tell the sandbox to let it go, it fails anyway.

It is fine if we end up having to add some more of those.

[strogdon]

I do have freefont installed, kpathsea and rebuilds are still building.

[kiwifb]

The doc build has gone back to be mostly single threaded I think. Parallelism has to be controlled with SAGE_NUM_THREADS and SAGE_NUM_THREADS_PARALLEL. I will try to introduce them in the environment in this cycle to see what happens.

[strogdon]

After upgrading kpathsea I still have a sandbox violation:

LaTeX Warning: Citation `bibliography:pyt' on page 88 undefined on input line 6
366.

 [6][88] * ACCESS DENIED:  mkdir:              /var/cache/fonts/pk/modeless/public
 * ACCESS DENIED:  mkdir:              /var/cache/fonts/pk/modeless/public
mkdir: cannot create directory ‘././var/cache/fonts/pk/modeless/public’: Permission denied
mktexpk: /usr/share/texmf-dist/web2c/mktexdir /var/cache/fonts/pk/modeless/public/tex-gyre failed.
kpathsea: Appending font creation commands to missfont.log.
dvipdfmx:warning: Could not locate a virtual/physical font for TFM "ec-qhvb".
dvipdfmx:warning: >> There are no valid font mapping entry for this font.
dvipdfmx:warning: >> Font file name "ec-qhvb" was assumed but failed to locate that font.
dvipdfmx:fatal: Cannot proceed without .vf or "physical" font for PDF output...

The ec-qhvb tfm font seems to be associated with the Japanese documentation. However there are situations where the font is generated correctly. As in

(/usr/share/texmf-dist/[9tex]/generic/iftex/ifvtex.sty (./a_tour_of_sage.aux
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 2+293/600 --dpi 1493 ec-qhvb
)

[kiwifb]

OK the sandbox violation seems to have been me removing VARTEXFONTS - unfortunately still needed when you deal with more than English and French. There is another failure lurking that I have not identified yet.

[kiwifb]

Document Class: sphinxmanual 2019/12/01 v2.3.0 Document class (Sphinx manual)
</usr/share/texmf-dist/fonts/type
1/public/amsfonts/cm/cmr10.pfb></usr/share/texmf-dist/fonts/type1/public/amsfon
ts/cm/cmsy10.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmsy5.pf
b(/usr/share/texmf-dist/tex/latex/base/report.c>ls
Document Class: report 2023/05/17 v1.4n Standard LaTeX document class

Output written on a_tour_of_sage.pdf (13 pages, 129696 bytes).
(/usr/share/texmf-dist/tex/latex/base/size10.clomktexpk: `gsftopk upgbm-h 1436' failed to make upgbm-h.1436pk.

Package fancyhdr Warning: \headheight is too small (12.0pt):
(fancyhdr)                Make it at least 15.0pt, for example:
(fancyhdr)                \setlength{\headheight}{15.0pt}.
(fancyhdr)                You might also make \topmargin smaller to compensate:

(fancyhdr)                \addtolength{\topmargin}{-3.0pt}.

[39]kpathsea: Appending font creation commands to missfont.log.
dvipdfmx:warning: Could not locate a virtual/physical font for TFM "upgbm-h".
dvipdfmx:warning: >> There are no valid font mapping entry for this font.
dvipdfmx:warning: >> Font file name "upgbm-h" was assumed but failed to locate that font.
dvipdfmx:fatal: Cannot proceed without .vf or "physical" font for PDF output...

No output PDF file written.
Transcript written on a_tour_of_sagLatexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  dvipdf: Command for 'dvipdf' gave return code 256

Latexmk: If appropriate, the -f option can be used to get latexmk
  to try to force complete processing.
e.log.
make[2]: *** [Makefile:30: a_tour_of_sage.pdf] Error 12
make[2]: Leaving directory '/home/portage/sci-mathematics/sage-doc-9999/work/build_doc/latex/ja/a_tour_of_sage'

probably mixed with other stuff from parallel processing.

[kiwifb]

$ mktexpk --mfmode / --bdpi 600 --mag 2+236/600 --dpi 1436 upgbm-h
mktexpk: Running gsftopk upgbm-h 1436
gsftopk(k) version 1.19.2
render.ps: font HaranoAjiGothic-Medium-UniJIS2004-UTF16-H is not defined.
GPL Ghostscript 10.02.1: Unrecoverable error, exit code 1
/10021
gs terminated abnormally with status 1

grep: upgbm-h.log: No such file or directory
mktexpk: `gsftopk upgbm-h 1436' failed to make upgbm-h.1436pk.

feels like a bug

$ cat /usr/share/texmf-dist/fonts/map/dvipdfmx/ptex-fontmaps/haranoaji/uptex-haranoaji-04.map
urml    UniJIS2004-UTF16-H      HaranoAjiMincho-Regular.otf
urmlv   UniJIS2004-UTF16-V      HaranoAjiMincho-Regular.otf
ugbm    UniJIS2004-UTF16-H      HaranoAjiGothic-Medium.otf
ugbmv   UniJIS2004-UTF16-V      HaranoAjiGothic-Medium.otf
uprml-h UniJIS2004-UTF16-H      HaranoAjiMincho-Regular.otf
uprml-v UniJIS2004-UTF16-V      HaranoAjiMincho-Regular.otf
upgbm-h UniJIS2004-UTF16-H      HaranoAjiGothic-Medium.otf
upgbm-v UniJIS2004-UTF16-V      HaranoAjiGothic-Medium.otf
uprml-hq        UniJIS-UCS2-H   HaranoAjiMincho-Regular.otf
upgbm-hq        UniJIS-UCS2-H   HaranoAjiGothic-Medium.otf

and upgbm-h.tfm is on the system, and the appropriate .map file is present.

[kiwifb]

And the issue seems to only be in Japanese. All other languages work.

[strogdon]

I tried vanilla with -j1 and got the failure

grep: upgbm-h.log: No such file or directory
mktexpk: `gsftopk upgbm-h 1436' failed to make upgbm-h.1436pk.
kpathsea: Appending font creation commands to missfont.log. 
dvipdfmx:warning: Could not locate a virtual/physical font for TFM "upgbm-h".
dvipdfmx:warning: >> There are no valid font mapping entry for this font.
dvipdfmx:warning: >> Font file name "upgbm-h" was assumed but failed to locate that font.
dvipdfmx:fatal: Cannot proceed without .vf or "physical" font for PDF output...

No output PDF file written.
Collected error summary (may duplicate other messages):
  dvipdf: Command for 'dvipdf' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.
Latexmk: Errors, so I did not complete making targets
make[5]: *** [Makefile:30: tutorial-jp.pdf] Error 12
Error building the documentation.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/local/sage-git/sage/src/sage_docbuild/__main__.py", line 508, in <module>
    sys.exit(main())
             ^^^^^^
  File "/local/sage-git/sage/src/sage_docbuild/__main__.py", line 504, in main
    builder()
  File "/local/sage-git/sage/src/sage_docbuild/builders.py", line 299, in pdf
    raise RuntimeError(error_message % (command, tex_dir))
RuntimeError: failed to run $MAKE all-pdf in /local/sage-git/sage/local/share/doc/sage/latex/ja/tutorial

Not sure if something is missing in texlive or if there is a coding problem.

[kiwifb]

That's exactly the same issue that I have. I suspect something subtle in the texlive install for japanese.

[strogdon]

One thing is certainly incorrect

$ ls -al /usr/share/texmf-dist/dvipdfmx/
total 2
drwxr-xr-x  2 root root 1024 Dec 27 16:38 .
drwxr-xr-x 30 root root 1024 Dec 27 16:38 ..
lrwxrwxrwx  1 root root   46 Dec 27 16:38 dvipdfmx.cfg -> '../../../../etc/texmf/dvipdfmx).d/dvipdfmx.cfg

The symlink should point to ../../../../etc/texmf/dvipdfmx.d/dvipdfmx.cfg.

[kiwifb]

This is interesting indeed!

[strogdon]

Vanilla builds with the appropriate fix.

[kiwifb]

I think it should go in bugzilla. I'll see if I can figure which package dvipdfmx.cfg belongs to. I think it is texlive-core but I am not 100% per cent certain.

[kiwifb]

texlive-core but I think the error may be a corner case in one of the eclass.

[kiwifb]

This is done by the texlive-common.eclass

 ^[[32m*^[[0m Moving (and symlinking) /usr/share/texmf-dist/web2c/updmap-hdr.cfg to /etc/texmf/web2c.d
 ^[[32m*^[[0m Moving (and symlinking) /usr/share/texmf-dist/web2c/fmtutil-hdr.cnf to /etc/texmf/web2c.d
 ^[[32m*^[[0m Moving (and symlinking) /usr/share/texmf-dist/dvipdfmx/dvipdfmx.cfg to /etc/texmf/dvipdfmx.d
>>> Completed installing app-text/texlive-core-2023 into /home/portage/app-text/texlive-core-2023/image

and they are all affected

fbissey@tarazed ~ $ ll /usr/share/texmf-dist/web2c/updmap-hdr.cfg
lrwxrwxrwx 1 root root 45 Dec 28 14:05 /usr/share/texmf-dist/web2c/updmap-hdr.cfg -> '../../../../etc/texmf/web2c).d/updmap-hdr.cfg'
fbissey@tarazed ~ $ ll /usr/share/texmf-dist/web2c/fmtutil-hdr.cnf
lrwxrwxrwx 1 root root 46 Dec 28 14:05 /usr/share/texmf-dist/web2c/fmtutil-hdr.cnf -> '../../../../etc/texmf/web2c).d/fmtutil-hdr.cnf'
fbissey@tarazed ~ $ ll /usr/share/texmf-dist/dvipdfmx/dvipdfmx.cfg
lrwxrwxrwx 1 root root 46 Dec 28 14:05 /usr/share/texmf-dist/dvipdfmx/dvipdfmx.cfg -> '../../../../etc/texmf/dvipdfmx).d/dvipdfmx.cfg'

[strogdon]

The links are fine for my texlive-2019. So the eclass has changed since it was installed (12/20/2023).

[kiwifb]

Found the error. texlive-common_handle_config_files function in texlive-common.eclass

Gentoo

dosym8 -r /etc/texmf/$(dirname ${f}).d/$(basename ${f}) ${TEXMF_PATH}/${f}

Texlive-overlay

dosym -r "/etc/texmf/${rel_dir}).d/$(basename "${f}")" "${texmf_path}/${f}"

Note the stray )

[kiwifb]

Added a comment to the texlive 2023 bug https://bugs.gentoo.org/836779

[kiwifb]

Fixing the eclass and re-installing texlive-core-2023 fixed the links. Next step, all languages sage-doc build.

[strogdon]

I was able to build the docs for all langs.

[kiwifb]

How long does it take on your system? I am doing around 30mn for English + French and I estimate I will do 35 mn for all languages (based on 34mn for everything but Japanese). This is up from 20mn for English + French before the switch to luatex. So, it is slower.

[kiwifb]

Success in 33mn and 54s. Keeping the issue open until we have a fix in the texlive overlay.

[strogdon]

How time this? I've not done any timings.

[kiwifb]

genlop -t sage-doc from the gentoolkit I think.

[strogdon]

Building all langs:
merge time: 53 minutes and 6 seconds.

and

 Tue Dec 19 16:59:03 2023 >>> sci-mathematics/sage-doc-9999
   merge time: 31 minutes.

So, yes the build seems to take longer.

[kiwifb]

https://gitweb.gentoo.org/proj/tex-overlay.git/commit/eclass/texlive-common.eclass?h=main&id=02a45ff97b50eb0ecffa7532bf1697b065b71925 I will officially close this issue