Build fails for release and forks, advice?

[tetov]

Hi!

I understand that the status of sphinxcontrib-versioning is a bit complicated, but I gather from reading from the issues and PRs and from looking at a few doc setups that people have this working. I was wondering if someone could help me with an issue I haven't found reported here on the repo.

Running sphinx-versioning build with the pypi version I get the error documented in issue #66 fixed in PR #69. However, running @z00sts' PR branch quits with a file not found error.

I've also tried @leokoppel's fork, but there it hangs after two runs of doc building and no files found in the reported path (/tmp/tmp*sphinxcontrib_versioning).

Anyone encountered something similar or could give some points towards getting this to work with compas-dev/compas?

General info

$ python --version
Python 3.6.9

$ pip freeze | grep -i sphinx
nbsphinx==0.4.3
Sphinx==2.2.0
sphinx-compas-theme==0.4.1
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
sphinxcontrib-versioning==2.2.1

$ git remote -v

origin          git@github.com:compas-dev/compas.git (fetch)
origin          git@github.com:compas-dev/compas.git (push)

Logs

$ pip install git+https://github.com/z00sts/sphinxcontrib-versioning.git@python3.6-import-error-workaround
$ sphinx-versioning -L build -i -r master docs /dist/docs
=> Gathering info about the remote git repository...
=> Getting list of all remote branches/tags...
=> Found: docs-pull-req footnote-fix master patch-1 pathlib_support quick-fix-clr-pythonnet sphinx-versioning update_robot worbit-patch-1 0.1.0 0.1.0-drx_numpy_callback 0.1.0-facenetwork_import 0.1.0-subd_ck_example_imports 0.1.0-viewer_glut_title_str 0.1.1 0.1.1-import_error_smoothing_cpp 0.1.1-import_problems_cpp_versions 0.1.1-leaves_after_embedding 0.1.1-rhino_utility_aliases 0.1.1-xfunc_callback_args 0.1.1_ga_bugfix v0.3.0 v0.3.1 v0.3.2 v0.3.3 v0.3.4 v0.3.5 v0.3.6 v0.4.0 v0.4.1 v0.4.10 v0.4.11 v0.4.12 v0.4.13 v0.4.14 v0.4.15 v0.4.16 v0.4.17 v0.4.18 v0.4.19 v0.4.2 v0.4.20 v0.4.21 v0.4.22 v0.4.23 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.4.7 v0.4.8 v0.4.9 v0.5.0 v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.6.2 v0.7.0 v0.7.1 v0.7.2
=> With docs: docs-pull-req footnote-fix master patch-1 pathlib_support quick-fix-clr-pythonnet sphinx-versioning update_robot worbit-patch-1 v0.3.0 v0.3.1 v0.3.2 v0.3.3 v0.3.4 v0.3.5 v0.3.6 v0.4.0 v0.4.1 v0.4.10 v0.4.11 v0.4.12 v0.4.13 v0.4.14 v0.4.15 v0.4.16 v0.4.17 v0.4.18 v0.4.19 v0.4.2 v0.4.20 v0.4.21 v0.4.22 v0.4.23 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.4.7 v0.4.8 v0.4.9 v0.5.0 v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.6.2 v0.7.0 v0.7.1 v0.7.2
=> Root ref is: master
=> Pre-running Sphinx to collect versions' master_doc and other info.
usage: sphinx-versioning [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]
sphinx-versioning: error: cannot find files ['/tmp/tmpm9pm2hi8sphinxcontrib_versioning']
=> sphinx-build failed for branch/tag: master
Failure.

$ pip install git+https://github.com/leokoppel/sphinxcontrib-versioning
$ sphinx-versioning -L build -i -r master docs /dist/docs
=> Gathering info about the remote git repository...
=> Getting list of all remote branches/tags...

[...]
# Runs twice with Sphinx-warnings that we see with normal runs
[...]

copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 27 warnings.

The HTML pages are in ../../../../tmp/tmplibvh55lsphinxcontrib_versioning.

# Freezes

[super-sk]

Hi Anton !

I encountered the same problem with the sphinxcontrib-versioning 2.2.1 from pypi. Even if I solved my problem in #66 I'm still stuck like you mentioned above.

[Stonesjtu]

Hi there, after digging around for 1 hour, I found a simple hack should work for this case.

https://github.com/sphinx-contrib/sphinxcontrib-versioning/blob/86a28dc92ce3ff42ed36a8b3e4dc6bdcff90f2c4/sphinxcontrib/versioning/sphinx_.py#L202

Change the argv to argv[1:].

Currently by default argv has three elements: sphinx-build, {SOURCE}, {TARGET}. But I suspect the new sphinx accepts only the last two, it will treat {TARGET} as filename option and check if it's really a file, which is a folder of course (Here's the logging line Not files).

[tetov]

Thanks! Going to test it out later today!

[Stonesjtu]

@tetov sorry for post a wrong solution, it's for another bug. BTW I encountered the same freezing bug too.

[Stonesjtu]

There are several things to make it work in my setting up.

1. add sphinxcontrib.versioning.sphinx_ to the extension list in conf.py
2. check-out #46 to fix the version mis-match issue.

It should work now.

[vancheese]

I've used fresh install from the main by following the two points above (I manually patched the sphinx.py) and then also also the ImportError: cannot import name 'build_main' fix The program hangs after display

The HTML pages are in /tmp/tmpbuild directory

This error occurs with my real project and also with the tutorials simple example (https://sphinxcontrib-versioning.readthedocs.io/en/latest/tutorial.html#build-all-versions)

[Stonesjtu]

You can try add some log lines in these callbacks to see if this extension is registered correctly. https://github.com/sphinx-contrib/sphinxcontrib-versioning/blob/86a28dc92ce3ff42ed36a8b3e4dc6bdcff90f2c4/sphinxcontrib/versioning/sphinx_.py#L47 https://github.com/sphinx-contrib/sphinxcontrib-versioning/blob/86a28dc92ce3ff42ed36a8b3e4dc6bdcff90f2c4/sphinxcontrib/versioning/sphinx_.py#L65

[GlitchyPSIX]

Any progress in this? I have this same problem.

[vancheese]

You can try add some log lines in these callbacks to see if this extension is registered correctly. https://github.com/sphinx-contrib/sphinxcontrib-versioning/blob/86a28dc92ce3ff42ed36a8b3e4dc6bdcff90f2c4/sphinxcontrib/versioning/sphinx_.py#L47

https://github.com/sphinx-contrib/sphinxcontrib-versioning/blob/86a28dc92ce3ff42ed36a8b3e4dc6bdcff90f2c4/sphinxcontrib/versioning/sphinx_.py#L65

I added simple print commands to see if the two functions were called and finished. Both function weren't called before the programm freezed

[Stonesjtu]

@vancheese Then the extension itself is not registered properly. Can you post the conf.py and command that failed?

[vancheese]

I've identified where the program freeze In sphinx_.py in read_config, the program stops at

config = queue.get()

[GlitchyPSIX]

I've found that applying the patch from #46, then applying the argv[1:] fix and #66 makes the build system work and not stall anymore...

This may be because of sphinx version uncompatibility, because the fixes themselves end up stalling until #46 is applied

[paddy-hack]

I've been hit by this as well and tried to figure out what broke in which sphinx versions. I've used the Docker sphinx that I maintain for this and used the "quick-and-dirty" documentation from the tutorial for testing.

• for 1.6.5 to work, I had to apply #46
• for 1.7.0 to work, I had to apply #69 as well as the argv[1:] fix mentioned above in addition to #46
• for 1.8.0 to work, I had to apply #57 in addition to the changes for 1.7.0 and 1.6.5
• for 2.0.0 to work, I had to apply #57 in addition to the changes for 1.7.0 and 1.6.5

Version 2.0.1 is the first version where I observed the hang/freeze. Apart from the sphinx version, there are no other differences between my Docker images for 2.0.0 and 2.0.1. The changes between those two don't provide any clue, to me at least, what might have triggered this.

[GlitchyPSIX]

@paddy-hack The changes I spoke about in my last comment work on sphinx 2.2.1 too

[dlopezlo]

Hi,

i'm using the fork: https://github.com/Smile-SA/sphinx-versions installed from pip and all of the above patches are already applied but it still hangs from sphinx-2.0.1.

Have anyone tried this library?

@GlitchyPSIX does it work for you with patches you mentioned? #46 and #66

[GlitchyPSIX]

@dlopezlo Yes (sphinx 2.2.1), I have managed to get sphinxcontrib-versioning building properly by applying #46, #66, and the argv[1:] fix (I did them in this order)

Do remember to add the sphinxcontrib.versioning.sphinx_ extension in conf.py

I made a separate conf.py for when I want to build separately without versioning which doesn't have that extension which I build using sphinx-build

[paddy-hack]

With some real documentation, I also ran into #48, which is fixed in #51.

[paddy-hack]

As for adding the sphinxcontrib.versioning.sphinx_ extension in conf.py, while that may be an obvious(?) thing to do, I didn't and it is not in the documentation, yet :cry:.
I'm off now giving that a try :crossed_fingers:

[paddy-hack]

Adding sphinxcontrib.versioning.sphinx_ to the extensions array in conf.py fixes the hang with this caveat that you can only build documentation for versions that do. If you try to build (older) versions of your documentation where the extension has not been added, the build will hang.

Fix confirmed to work on 2.0.1, 2.1.0, 2.1.2, 2.2.0 and 2.2.1 using the tutorial example with the extension added to conf.py.

[fmmix]

After applying all those fixes I get the message:

miniconda3/envs/py35/lib/python3.5/site-packages/sphinxcontrib/versioning/sphinx_.py", line 101, in html_page_context
    cls.VERSIONS.context = context
AttributeError: 'NoneType' object has no attribute 'context'

Does anyone have the same issue? Or did I maybe do something wrong?

[GlitchyPSIX]

Adding sphinxcontrib.versioning.sphinx_ to the extensions array in conf.py fixes the hang with this caveat that you can only build documentation for versions that do. If you try to build (older) versions of your documentation where the extension has not been added, the build will hang.

My workaround for this is just a second conf.py in a separate directory made only for non-versioned building

[paddy-hack]

I'm aware of your work around but that doesn't solve thing for documentation versions that have already been committed and pushed :cry:

[paddy-hack]

@fmmix I got the same error when accidently running sphinx-build instead of sphinx-versioning.

[paddy-hack]

@GlitchyPSIX I've resorted to conditionally added the extension based on an environment variable. Since I normally only want to run sphinx-versionsing in CI (and sphinx-build during "development" of the documentation), I have added it like so

if 'CI' in os.environ and os.environ['CI'] == 'true':
    extensions.append('sphinxcontrib.versioning.sphinx_')

Works For Me :smile:

[GlitchyPSIX]

@paddy-hack Haha, yeah, figured this would be a solution

In the end that's kind of what I achieved with the second conf.py since I don't use CI but batch files to generate the respective documentation types

Maybe your solution can work for others facing this issue? Looks a bit more catch-all than my local-based case.

[ClementMaliet]

It took me a while but I finally figured it out for good.

I will not make a PR because I find the fix to be especially ugly but I leave it there as a git patch for those encountering the issue.

The bug happens starting from Sphinx 2.0.1 as some noticed and is due to the fact that the CONFIG_FILENAME attribute of the sphinx.application module was deprecated with a 3.0.0 deletion in mind. Now the Sphinx deprecation helper actually wrap the entire module in sys.modules into a _ModuleWrapper to wrap the module __getattribute__ to send a DeprecationWarning if the deprecated attribute is to be fetch. This is all good and all but it does breaks the attempt that sphinxcontrib.versioning makes to monkey patch the sphinx.application.Config object to auto-register sphinxcontrib.versioning.sphinx_ as an extension (which actually fixes the hang as previous comments found out) upon instantiation.

My UGLY hotfix bypasses the _ModuleWrapper explicitly and will not work for sphinx versions before the 2.0.1 and after at least the 3.0.0.

Index: sphinxcontrib/versioning/sphinx_.py
<+>UTF-8
===================================================================
--- sphinxcontrib/versioning/sphinx_.py (revision f286d43595d75c08cb51a0153a9cdf3b8a1957b3)
+++ sphinxcontrib/versioning/sphinx_.py (date 1582638010030)
@@ -184,8 +184,13 @@
     :param str current_name: The ref name of the current version being built.
     :param bool is_root: Is this build in the web root?
     """
-    # Patch.
-    application.Config = ConfigInject
+    # Update monkey patch to work with Sphinx 2.0.1+ _ModuleWrapper addition around sphinx.application due to
+    # CONFIG_FILENAME deprecation.
+    # _ModuleWrapper does not redirect __setattr__ calls to the underlying module so we need to bypass it to inject
+    # the extension.
+    # This is ugly and will break starting from at least Sphinx 3.0+ but the idea of monkey patching the extension
+    # silently into sphinx was hackish at most to begin with.
+    application._module.Config = ConfigInject  # noqa
     if config.show_banner:
         EventHandlers.BANNER_GREATEST_TAG = config.banner_greatest_tag
         EventHandlers.BANNER_MAIN_VERSION = config.banner_main_ref

[Holzhaus]

I needed a versioning tool for Sphinx but all existing projects that I found didn't work properly and their code was messy, so I decided to implement my own solution and published it here: https://holzhaus.github.io/sphinx-multiversion/master/index.html