sphinx-build -jauto has nondeterministic html results

[bmwiedemann]

Describe the bug html rendering with -jauto has nondeterministic results, except when run on a 1-core-VM

To Reproduce Steps to reproduce the behavior:

osc checkout openSUSE:Factory/kernel-docs && cd $_
osc build --noservice
# extract html/Documentation/output/core-api/refcount-vs-atomic.html
# compare to those from another build

or maybe

git clone --depth 1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
cd linux-2.6
make htmldocs

Expected behavior output should be deterministic

Your project https://github.com/torvalds/linux/blob/master/Documentation/conf.py

Screenshots

-<p>Il kernel è scritto nel linguaggio di programmazione C <a class="reference internal" href="../../../process/programming-language.html#href_anchor" id="a_idN"><span>[c-language]</span></a>.
-Più precisamente, il kernel viene compilato con <code class="docutils literal notranslate"><span class="pre">gcc</span></code> <a class="reference internal" href="../../../process/programming-language.html#href_anchor" id="a_idN"><span>[gcc]</span></a> usando
-l’opzione <code class="docutils literal notranslate"><span class="pre">-std=gnu89</span></code> <a class="reference internal" href="../../../process/programming-language.html#href_anchor" id="a_idN"><span>[gcc-c-dialect-options]</span></a>: il dialetto GNU
+<p>Il kernel è scritto nel linguaggio di programmazione C <a class="reference internal" href="#href_anchor" id="a_idN"><span>[c-language]</span></a>.
+Più precisamente, il kernel viene compilato con <code class="docutils literal notranslate"><span class="pre">gcc</span></code> <a class="reference internal" href="#href_anchor" id="a_idN"><span>[gcc]</span></a> usando
+l’opzione <code class="docutils literal notranslate"><span class="pre">-std=gnu89</span></code> <a class="reference internal" href="#href_anchor" id="a_idN"><span>[gcc-c-dialect-options]</span></a>: il dialetto GNU

Environment info

• OS: openSUSE Tumbleweed
• Python version: 3.7.3
• Sphinx version: 2.2.0
• Sphinx extensions: websupport, devhelp, htmlhelp, jsmath, serializinghtml

Additional context Got same result when adding -E

This bug was found while working on reproducible builds for openSUSE.

[bmwiedemann]

This is still an issue with Sphinx-7.2.6

So far affected are 3 openSUSE packages: kernel-docs qemu guake

One simple reproducer is https://github.com/Guake/guake/blob/master/docs/Makefile where I get varying results from

make -O -j4 -C docs html man

even without -jauto and with -E

[bmwiedemann]

I found a workaround for guake: using different -d doctrees paths helps there.

[bmwiedemann]

This is likely a duplicate of #2946

[bmwiedemann]

Sphinx 7.4.5 still causes non-determinism in our kernel-docs build with -jauto from https://github.com/torvalds/linux/blob/c763c4339688/Documentation/sphinx/parallel-wrapper.sh

[jayaddison]

This is still an issue with Sphinx-7.2.6

So far affected are 3 openSUSE packages: kernel-docs qemu guake

One simple reproducer is https://github.com/Guake/guake/blob/master/docs/Makefile where I get varying results from

make -O -j4 -C docs html man

even without -jauto and with -E

@bmwiedemann would it be possible to prepare an even-more-minimal repro example to replicate the problem? The reason I ask is that I'm aware of a separate race-condition issue ( https://github.com/Guake/guake/issues/2219 ) that occurs when building guake using that same Makefile, and I want to try to eliminate that so that we can more easily determine the root cause in Sphinx.

[bmwiedemann]

I think, there may be two different issues.

1. There is #2946 which covers the guake case - there is parallelism external to sphinx and two different sphinx processed don't take care to play nicely with the shared doctrees cache files.

2. There is kernel-docs where a single sphinx -jauto call does all the processing. The latest diff is https://rb.zq1.de/compare.factory-20240904/diffs/kernel-source-compare.out - this one would probably the better base for building a reproducer. Not sure, I will have time for it soon.

[jayaddison]

2. There is kernel-docs where a single sphinx -jauto call does all the processing. The latest diff is https://rb.zq1.de/compare.factory-20240904/diffs/kernel-source-compare.out - this one would probably the better base for building a reproducer.

From inspecting the diff: most, perhaps all, of the differences appear to be related to the toctree.

[jayaddison]

@bmwiedemann I've drafted #12870 that I believe may help for this -- but I don't yet have a reliable way to minimally replicate the problem here. Would you be able to test that changeset to check whether it helps?