search

sphinx-doc / sphinx

The Sphinx documentation generator
https://www.sphinx-doc.org/
Other
6.23k stars 2.04k forks source link

Code triggers `assert parentSymbol` after seemingly irrelevant changes? #10220

Open iquah1 opened 2 years ago

iquah1 commented 2 years ago

Describe the bug

So I have a build running where everything is going fine. When I add a new file which is rather simple (along the lines of the following)

Test
====

Some text

and I run the build, I run into the assertion error from above.

How to Reproduce

The working build

$ git clone git@github.com:openfheorg/openfhe-temp.git
$ cd openfhe-temp
$ git checkout documentation
$ mkdir build && cd build && cmake .. && make Sphinx

which should generate everything just fine. Now, check out this other branch, documentation_broken. Now, clone the branch where it is breaking

$ git checkout documentation_broken
$ cd .. && rm -r build && mkdir build && cd build && cmake .. && make Sphinx

and you'll see that it is breaking

Expected behavior

Does not run into the issue

Your project

https://github.com/openfheorg/openfhe-temp

Screenshots

No response

OS

Linux

Python version

3.9

Sphinx version

4.2

Sphinx extensions

autodoc, doctest, mathjax, viewcode, imgmath, duration, todo

Extra tools

breathe

Additional context

You can see the diff between documentation and documentation_broken here. Any insight into the issue would be greatly appreciated.

The error log:

# Sphinx version: 4.4.0
# Python version: 3.6.13 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 3.0.2
# Last messages:
#   done
#   
#   writing output... [ 20%] assets/sphinx_rsts/contributing/contributing .. assets/sphinx_rsts/intro/get_started
#   
#   writing output... [ 40%] assets/sphinx_rsts/intro/installation/cmake_in_palisade .. assets/sphinx_rsts/intro/resources
#   
#   writing output... [ 60%] assets/sphinx_rsts/intro/simple .. assets/sphinx_rsts/misc/publications
#   
#   writing output... [ 80%] assets/sphinx_rsts/modules/binfhe .. assets/sphinx_rsts/modules/core/math/core_math_bigintnat
#   
# Loaded extensions:
#   sphinx.ext.mathjax (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/alabaster/__init__.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.doctest (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/doctest.py
#   sphinx.ext.viewcode (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/viewcode.py
#   sphinx.ext.imgmath (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/imgmath.py
#   sphinx.ext.todo (4.4.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/ext/todo.py
#   breathe (4.32.0) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/breathe/__init__.py
#   sphinx_rtd_theme (unknown version) from /home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx_rtd_theme/__init__.py
Traceback (most recent call last):
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/cmd/build.py", line 284, in build_main
    app.build(args.force_all, filenames)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/application.py", line 337, in build
    self.builder.build_update()
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 296, in build_update
    len(to_build))
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 358, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 530, in write
    nproc=self.app.parallel - 1)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 566, in _write_parallel
    doctree = self.env.get_and_resolve_doctree(docname, self)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 535, in get_and_resolve_doctree
    self.apply_post_transforms(doctree, docname)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 581, in apply_post_transforms
    transformer.apply_transforms()
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/transforms/__init__.py", line 87, in apply_transforms
    super().apply_transforms()
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/transforms/post_transforms/__init__.py", line 43, in apply
    self.run(**kwargs)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/transforms/post_transforms/__init__.py", line 93, in run
    typ, target, node, contnode)
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/domains/cpp.py", line 7933, in resolve_xref
    target, node, contnode)[0]
  File "/home/iquahduality/anaconda3/envs/palisade/lib/python3.6/site-packages/sphinx/domains/cpp.py", line 7816, in _resolve_xref_inner
    assert parentSymbol  # should be there
AssertionError
jakobandersen commented 2 years ago

Thanks for reporting. I can reproduce the problem locally (though with an extra cmake ../ in the build/ folder). It is a rather large project so I'm not completely sure what is going on, but my gut-feeling is that there may be a bad interaction between having duplicate C++ declarations and then some xref lookup code. This should of course not result in an assertion. I don't have much time right now, but if possible it would be great if you could experiment with making a smaller version that reproduces the problem. Then I'll take a deeper look at it as soon as possible. The name where it goes wrong in my test is bigintnat::NativeIntegerT::Integer. It looks like there are NativeIntegerT declarations in both assets/sphinx_rsts/modules/core/math/core_math_bigintnat and assets/sphinx_rsts/modules/core/math/core_math, so perhaps a brutal limitation to those two files (and maybe just a small part of them).

iquah1 commented 2 years ago

though with an extra cmake ../ in the build/ folder

Ahh, so sorry, I've edited it to include those lines in case someone else takes a look too.

duplicate C++ declarations and then some xref lookup code

Gotcha!

The name where it goes wrong in my test is bigintnat::NativeIntegerT::Integer. It looks like there are NativeIntegerT declarations in both assets/sphinx_rsts/modules/core/math/core_math_bigintnat and assets/sphinx_rsts/modules/core/math/core_math

You got it! Out of curiosity how did you narrow it down to those two? When I run it I get between 5 to 10K lines of messages so it's hard to parse through all of them

jakobandersen commented 2 years ago

The name where it goes wrong in my test is bigintnat::NativeIntegerT::Integer. It looks like there are NativeIntegerT declarations in both assets/sphinx_rsts/modules/core/math/core_math_bigintnat and assets/sphinx_rsts/modules/core/math/core_math

You got it! Out of curiosity how did you narrow it down to those two? When I run it I get between 5 to 10K lines of messages so it's hard to parse through all of them

The output is indeed rather large. I get close to 13k lines :-). This assertion has triggered before so I have inserted som extra debug output to help diagnose it. The situation is that it is trying to resolve an xref and first needs to find the symbol for the current scope, but for some reason that lookup fails. The first line of the debug output starts with Target:, which is the xref to resolve, and then a line with ParentKey: which is a specification of the current scope to start lookup from. Then follows 9k lines with a dump of the complete symbol tree for the C++ domain. I thus found

Target:  NativeInt
ParentKey:  [(<ASTNestedNameElement>, None, '_CPPv49bigintnat'), (<ASTNestedNameElement>, <ASTTemplateParams>, '_CPPv4I0EN9bigintnat14NativeIntegerTE'), (<ASTNestedNameElement>, None, '_CPPv4N9bigintnat14NativeIntegerT7IntegerE')]

Edit: of which the key actually is

ParentKey:  [(bigintnat, None, _CPPv49bigintnat), (NativeIntegerT, template<typename NativeInt> , _CPPv4I0EN9bigintnat14NativeIntegerTE), (Integer, None, _CPPv4N9bigintnat14NativeIntegerT7IntegerE)]

and a bit of grepping in the symbol tree resulted in the lines

      NativeIntegerT: template<typename I> bigintnat::NativeIntegerT    (assets/sphinx_rsts/modules/core/math/core_math)                                                
      NativeIntegerT: template<typename NativeInt> bigintnat::NativeIntegerT : public lbcrypto::BigIntegerInterface<NativeIntegerT<NativeInt>>  (assets/sphinx_rsts/modu
les/core/math/core_math_bigintnat)

One guess is that the finds the wrong NativeIntegerT, but I'm not sure. Hence the request for a cut down version where the complete symbol tree is more manageable :-)

iquah1 commented 2 years ago

I've made the subset of the documentation where I only include the two files you mentioned! Notably, it still builds without error-ing out so I'm not sure what's up.

I then added back in all the files that should trigger an error (because of lots of conflicts) here, in a later commit but it's still successfully building.

I'd love your insight into this if you have any