multiprocessing error in theme_comparison.py on Windows

[2bndy5]

As advised by the contributing guidelines, I ran the theme_comparison.py script to do a final visual inspection... I was met with an error when executing:

python3 theme_comparison.py

full traceback

``` # Sphinx version: 4.5.0 # Python version: 3.11.2 (CPython) # Docutils version: 0.16 release # Jinja2 version: 3.1.2 # Last messages: # Loaded extensions: Traceback (most recent call last): File "path\to\.env\Lib\site-packages\sphinx\cmd\build.py", line 272, in build_main app = Sphinx(args.sourcedir, args.confdir, args.outputdir, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "path\to\.env\Lib\site-packages\sphinx\application.py", line 223, in __init__ self.setup_extension(extension) File "path\to\.env\Lib\site-packages\sphinx\application.py", line 380, in setup_extension self.registry.load_extension(self, extname) File "path\to\.env\Lib\site-packages\sphinx\registry.py", line 438, in load_extension metadata = setup(app) ^^^^^^^^^^ File "path\to\.env\Lib\site-packages\sphinx_immaterial\__init__.py", line 261, in setup app.setup_extension("sphinx_immaterial.postprocess_html") File "path\to\.env\Lib\site-packages\sphinx\application.py", line 380, in setup_extension self.registry.load_extension(self, extname) File "path\to\.env\Lib\site-packages\sphinx\registry.py", line 438, in load_extension metadata = setup(app) ^^^^^^^^^^ File "path\to\.env\Lib\site-packages\sphinx_immaterial\postprocess_html.py", line 61, in setup manager = multiprocessing.Manager() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\context.py", line 57, in Manager m.start() File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\managers.py", line 563, in start self._process.start() File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\process.py", line 121, in start self._popen = self._Popen(self) ^^^^^^^^^^^^^^^^^ File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\context.py", line 336, in _Popen return Popen(process_obj) ^^^^^^^^^^^^^^^^^^ File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 158, in get_preparation_data _check_not_importing_main() File "%APPDATA%\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 138, in _check_not_importing_main raise RuntimeError(''' RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. ```

The error message suggests using freeze_support() in the script's main function/entrypoint:

--- a/theme_comparison.py
+++ b/theme_comparison.py
@@ -13,6 +13,7 @@ Those requirements can be installed with:
 """
 import argparse
 from pathlib import Path
+from multiprocessing import freeze_support

 from sphinx.cmd.build import build_main

@@ -148,4 +149,6 @@ def build_docs(name, branch):
     print('')

-run_with_all_themes(build_docs)
+if __name__ == "__main__":
+    freeze_support()
+    run_with_all_themes(build_docs)

This solved the problem. I'm having trouble reproducing the issue now, so maybe there was something else at play.

[mgeier]

Hmm, that's strange. The script doesn't use multiprocessing and by default Sphinx shouldn't use parallel processing either.

The problem seems to originate from there: https://github.com/jbms/sphinx-immaterial/blob/c7e5ec93450ad78f54da0fe2544862b8472aa71a/sphinx_immaterial/postprocess_html.py#L61-L63

The error message suggests using freeze_support()

But it also says this:

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

I guess you are not using a frozen app?

I'm having trouble reproducing the issue now

Does that mean the suggested fix is not necessary?

[2bndy5]

I guess you are not using a frozen app?

AFAIK, theme_comparison.py was the app.

The problem seems to originate from there: https://github.com/jbms/sphinx-immaterial/blob/c7e5ec93450ad78f54da0fe2544862b8472aa71a/sphinx_immaterial/postprocess_html.py#L61-L63

It don't think the sphinx-immaterial theme is a good fit for theme_comparison.py because it heavily monkey-patches Sphinx during python runtime.

I'm not sure how theme_comparison assembles the working tree, but if previously built docs are included in the cache used to generate the working tree, then that may have been my problem. I had a doc build using sphinx-immaterial in doc/_build when initially executing them_comparison.

Does that mean the suggested fix is not necessary?

Given the above discussion, probably not.