Regression in Sphinx 8.1.0: latexpdf: Build failures

andreasabel commented 1 month ago

Describe the bug

Our CI for building the PDF of the Agda user manual failed with the advent of sphinx-8.1.0: https://github.com/agda/agda/actions/runs/11278714539/job/31367805388#step:5:14

Downloading sphinx-8.1.0-py3-none-any.whl.metadata (6.4 kB)

! Extra }, or forgotten \endgroup.
\sbox  ...box {\color@setgroup #2\color@endgroup }

l.103 \begin{sphinxadmonition}{note}{Note:}

? 
! Emergency stop.
\sbox  ...box {\color@setgroup #2\color@endgroup }

l.103 \begin{sphinxadmonition}{note}{Note:}

!  ==> Fatal error occurred, no output PDF file produced!

The same workflow succeeds with sphinx-8.0.1: https://github.com/agda/agda/actions/runs/11275627170/job/31357520346#step:5:14

Downloading sphinx-8.0.2-py3-none-any.whl.metadata (6.2 kB)

How to Reproduce

Reproduced by our CI (linked above).

Environment Information

See CI run (linked above).

Sphinx extensions

Using `sphinx_rtd_theme`

Additional context

No response

AA-Turner commented 1 month ago

cc @jfbu

AA-Turner commented 1 month ago

@andreasabel can you confirm all the required LaTeX packages and ideally optional packages are installed?

jfbu commented 1 month ago

As the CI workflow helpfully provides "TeXLive details" it seems after comparing 8.0.2 build and 8.1.0 one that nothing has changed regarding the used LaTeX support. I will look at OP's repo but ideally I would need to get my hands on the Sphinx produced .tex or even better a zip of the whole Sphinx build/latex repertory...

Update: as I can not reproduce the issue, probably a zip of build directory for failing build would not help me much, as I would need to have LaTeX running in a modified way to have more trace info, which I won't be able to do at my locale.

jfbu commented 1 month ago

I cloned https://github.com/agda/agda went to doc/user-manual, added to my python venv the missing sphinxcontrib_jquery and executed make latexpdf using Sphinx 8.1.0. I do not reproduce the issue, the PDF builds fine at my locale. It may be thus a problem that perhaps 8.1.0 indirectly adds a LaTeX dependency. The agda repo is at master branch commit d0d880fdc8ce20f7da64473de9f29d57c163e4ab.

(sorry for French locale in reproduced output)

user-manual$ make clean latexpdf
Removing everything under '_build'...
Sphinx v8.1.0 en cours d'exécution
chargement des traductions [en]... fait
création du répertoire de sortie... fait
Conversion de  `source_suffix = ['.lagda.rst', '.rst']` en `source_suffix = {'.lagda.rst': 'restructuredtext', '.rst': 'restructuredtext'}`.
construction en cours [mo] : cibles périmées pour les fichiers po 0
écriture... 
construction [latex] : all documents
mise à jour de l'environnement : [nouvelle configuration] 72 ajouté(s), 0 modifié(s), 0 supprimé(s)
lecture des sources... [100%] tools/search-about
recherche des fichiers périmés... aucun résultat trouvé
Environnement de sérialisation... fait
vérification de la cohérence... fait
documents en préparation... fait
copie des ressources... 
copie des fichiers de support TeX... Écriture du résultat du modèle évalué dans /Users/xxxxxxxx/agda/doc/user-manual/_build/latex/make.bat
Écriture du résultat du modèle évalué dans /Users/xxxxxxxx/agda/doc/user-manual/_build/latex/Makefile
Écriture du résultat du modèle évalué dans /Users/xxxxxxxx/agda/doc/user-manual/_build/latex/latexmkrc
Écriture du résultat du modèle évalué dans /Users/xxxxxxxx/agda/doc/user-manual/_build/latex/latexmkjarc
fait
copie de fichiers supplémentaires...  unicode-symbols-sphinx.tex.txtfait
copie des ressources: fait
traitement de Agda.tex en cours... 
index
 overview
 getting-started/index

...

....

...

Output written on Agda.pdf (311 pages, 1504588 bytes).
Transcript written on Agda.log.
Latexmk: Getting log file 'Agda.log'
Latexmk: Examining 'Agda.fls'
Latexmk: Examining 'Agda.log'
Latexmk: Index file 'Agda.idx' was written
Latexmk: Log file says output to 'Agda.pdf'
Have index file 'Agda.idx', Agda.ind Agda
Latexmk: All targets (Agda.pdf) are up-to-date

make[1] : on quitte le répertoire « /Users/xxxxxxxx/agda/doc/user-manual/_build/latex »

jfbu commented 1 month ago

Your LaTeX is more recent than mine

(./Agda.tex
LaTeX2e <2024-06-01> patch level 2
L3 programming layer <2024-09-10>

versus mine

(./Agda.tex
LaTeX2e <2024-06-01> patch level 2
L3 programming layer <2024-08-30>

I am now updating it to retry the build.

OK my TeXLive 2024 is fully up-to-date as of today and I build Agda.pdf with no hick-ups... (which is bad because I can't much help, the error message is too obscure to conclude for sure).

andreasabel commented 1 month ago

Thanks for looking into this!

Our CI uses a detailed LaTeX dependency list found here: https://github.com/agda/agda/blob/d0d880fdc8ce20f7da64473de9f29d57c163e4ab/src/github/workflows/user_manual.yml#L44-L78 Rather than installing large package collections such as the recommended texlive-fonts-extra (https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.latex.LaTeXBuilder), I am just the minimal amount of needed packages to decrease CI time. So if the reason is that a package is missing, this could be fixed by adding a package, but I don't know which one, because

the error does not refer to any missing package
the changelog for 8.1.0 does not mention any additionally needed packages

I cannot produce the issue locally either (maybe because on my own machine I have a more complete LaTeX installation).

Our extra packages are these: https://github.com/agda/agda/blob/d0d880fdc8ce20f7da64473de9f29d57c163e4ab/doc/user-manual/conf.py#L139-L144

\usepackage{amsmath}
\usepackage{bbm}
\usepackage{mathtools}
\usepackage{stmaryrd}
\usepackage{pifont}
\usepackage{keystroke}
\input{unicode-symbols-sphinx.tex.txt}

jfbu commented 1 month ago

So if the reason is that a package is missing, this could be fixed by adding a package, but I don't know which one, because
* the error does not refer to any missing package

* the changelog for 8.1.0 does not mention any additionally needed packages
I cannot produce the issue locally either (maybe because on my own machine I have a more complete LaTeX installation).

Yes, this is annoying. We did not add dependencies at 8.1.0 as far as I recall. The error seems to arise at location of an admonition, which we touched at 7.4.0 not at 8.1.0 (again as far as I recall and from checking again the 8.1.0 CHANGES, patches were done this summer). However 8.1.0 does have a few LaTeX related changes. Nothing obviously in relation with the typically obscure LaTeX error message (but logically it should be the case). (LaTeX by default turns all tracing off so that error messages are usually only helpful to locate the line number, then we add tracing commands to try to understand better, but for this we need to have an environment reproducing the error).

jfbu commented 1 month ago

Is the build directory clean of previous LaTeX builds? (I guess so, only asking to be sure).

andreasabel commented 1 month ago

This new code in 8.1.0 shows up in the LaTeX error message: https://github.com/sphinx-doc/sphinx/pull/12704/files#diff-a1afaa45ce0d76f824256e401717ff4b8636c83cf1d3c4309443364b746d902cR340

     % 8.1.0 auto-drops extra space if no icon
    \sbox\z@{\@nameuse{sphinx#1TtlIcon}}%
    \ifdim\wd\z@>\z@

Can this be a clue?

jfbu commented 1 month ago

Can this be a clue?

Not to me ;-). \sbox is a kernel LaTeX command whose usage here is perfectly legit, as well as \@nameuse. A problem could arise if #1 (from sphinx#1TtlIcon you quote) injects something bad but usually an error of the type "not allowed in \csname" would show. The \color@setgroup, \color@endgroup originates in \sbox expansion.

$ latexdef sbox

\sbox:
macro:->\protect \sbox  

\sbox :
\long macro:#1#2->\setbox #1\hbox {\color@setgroup #2\color@endgroup }

The full Agda.log if available would perhaps allow to check what #1 (from sphinx#1TtlIcon you quote, which ends inside #2 here) is at that location.

Just in case maybe try to add fontawesome5 as LaTeX package dependency if you don't have it already. But if this solves it, it shows there is a bug somewhere in Sphinx which is supposed to work without it and which I tested at length without it.

jfbu commented 1 month ago

I have reproduced the error via letting LaTeX think fontawesome5 and fontawesome are both not present.

Thus, this is an error on my part somewhere, sorry about that. Now checking closer.

andreasabel commented 1 month ago

Ah, in the meanwhile I also just tried adding the fontawesome5 package and it indeed fixes the problem: https://github.com/agda/agda/pull/7554/files#r1797268343

Excellent!

jfbu commented 1 month ago

My bad sorry. I indeed completely forgot at some point to update a default hacky definition of \spx@faIcon which was an earlier way to not display icons in titles and triggers only in absence of both fontawesome5.sty and fontawesome.sty. A PR will follow. Thanks for report.

jfbu commented 1 month ago

As I comment at #13009, we do require on Ubuntu a dependency which includes fontawesome5... nevertheless I still consider my error as a regression because 7.4.0 carefully made sure things would work even without fontawesome5. Alas, a 8.1.0 patch changed the mechanism to avoid unneeded space in title bar if icons are not available, and an auxiliary in another LaTeX file should have been updated then, so PDF builds got broken on systems lacking both of fontawesome5 and fontawesome. So it is still regression of sorts, indeed.

sphinx-doc / sphinx