Draft: ToCTree env collector: disable read-parallelism

[jayaddison]

Feature or Bugfix

• Bugfix

Purpose

Bugreports #6714 and #12409 indicate that the table-of-contents collection process is not currently implemented in a way that guarantees deterministic resolution in the presence of parallelism.

Until such time as we can implement that, disable parallel reads for the TocTreeCollector so that the tree is resolved serially.

Detail

• Disable parallel reads for the TocTreeCollector by setting parallel_read_safe: False for it.

Relates

• May resolve #6714.
• May resolve #12409.

[bmwiedemann]

I tested that this fully helps with linux kernel-docs reproducibilty.

[kartben]

Would be great to get a rough idea of the potential impact on build speed (if any). Thanks!

[chrisjsewell]

@jayaddison you realise that this would literally mean it would be impossible to run sphinx in parallel 😅 This is a non-starter, you need to actually fix the problem

[jayaddison]

@jayaddison you realise that this would literally mean it would be impossible to run sphinx in parallel 😅 This is a non-starter, you need to actually fix the problem

Agreed; I didn't realize that disabling read-parallelism for a single extension also causes read-parallelism to be disabled for all other extensions within the same build.

Fortunately there is at least one testroot (test-toctree-glob) that exhibits the toctree parallel-build-non-determinism, so it may be possible to narrow in on the problem more easily now.

[jayaddison]

I tested that this fully helps with linux kernel-docs reproducibilty.

Thank you, @bmwiedemann :+1:

[bmwiedemann]

A quick benchmark run with our kernel-docs package showed that build time on a 4-core-VM went from 626 to 733 seconds. A 8-core-VM went from 552 to 788 s.