tox-dev / sphinx-autodoc-typehints

Type hints support for the Sphinx autodoc extension
MIT License
562 stars 106 forks source link

Error building docs with v1.10 #116

Closed jklaise closed 5 years ago

jklaise commented 5 years ago

My docs now fail to build with 1.10 (work with 1.9 or lower):

Exception occurred:
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx_autodoc_typehints.py", line 60, in get_annotation_args
    original = getattr(sys.modules[module], class_name)
KeyError: 'numpy'
The full traceback has been saved in /tmp/sphinx-err-e8ca7ci1.log, if you want to report the issue to the developers.

Here's the log file:

# Sphinx version: 2.1.2
# Python version: 3.7.3 (CPython)
# Docutils version: 0.14 
# Jinja2 version: 2.10.1
# Last messages:
#   failed
#   failed: No such config value: typehints_fully_qualified
#   loading intersphinx inventory from https://docs.python.org/objects.inv...
#   intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv
#   building [mo]: targets for 0 po files that are out of date
#   building [html]: targets for 61 source files that are out of date
#   updating environment:
#   61 added, 0 changed, 0 removed
#   reading sources... [  1%] api/alibi
#   reading sources... [  3%] api/alibi.confidence
# Loaded extensions:
#   sphinx.ext.mathjax (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.1) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.1) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (1.0.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.3) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/alabaster/__init__.py
#   sphinx.ext.autodoc (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.doctest (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/doctest.py
#   sphinx.ext.intersphinx (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/intersphinx.py
#   sphinx.ext.todo (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/todo.py
#   sphinx.ext.coverage (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/coverage.py
#   sphinx.ext.ifconfig (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/ifconfig.py
#   sphinx.ext.viewcode (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/viewcode.py
#   sphinx.ext.napoleon (2.1.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/napoleon/__init__.py
#   sphinx_autodoc_typehints (unknown version) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx_autodoc_typehints.py
#   sphinxcontrib.apidoc (0.3.0) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinxcontrib/apidoc/__init__.py
#   nbsphinx (0.4.2) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/nbsphinx.py
#   nbsphinx_link (1.2.0) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/nbsphinx_link/__init__.py
#   m2r (0.2.1) from /home/janis/.conda/envs/py37/lib/python3.7/site-packages/m2r.py
Traceback (most recent call last):
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/cmd/build.py", line 284, in build_main
    app.build(args.force_all, filenames)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/application.py", line 345, in build
    self.builder.build_update()
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 319, in build_update
    len(to_build))
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 332, in build
    updated_docnames = set(self.read())
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 438, in read
    self._read_serial(docnames)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 460, in _read_serial
    self.read_doc(docname)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 504, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/io.py", line 325, in read_doc
    pub.publish()
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/core.py", line 217, in publish
    self.settings)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/io.py", line 113, in read
    self.parse()
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/parsers.py", line 94, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 171, in run
    input_source=document['source'])
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2753, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2326, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2338, in explicit_construct
    return method(self, expmatch)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2081, in directive
    directive_class, match, type_name, option_presets)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2130, in run_directive
    result = directive_instance.run()
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/directive.py", line 150, in run
    documenter.generate(more_content=self.content)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 760, in generate
    self.document_members(all_members)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 681, in document_members
    check_module=members_check_module and not isattr)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 757, in generate
    self.add_content(more_content)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 495, in add_content
    for i, line in enumerate(self.process_doc(docstrings)):
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 463, in process_doc
    self.options, docstringlines)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/application.py", line 449, in emit
    return self.events.emit(event, *args)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx/events.py", line 103, in emit
    results.append(callback(self.app, *args))
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx_autodoc_typehints.py", line 368, in process_docstring
    annotation, fully_qualified=app.config.typehints_fully_qualified)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx_autodoc_typehints.py", line 110, in format_annotation
    args = get_annotation_args(annotation, module, class_name)
  File "/home/janis/.conda/envs/py37/lib/python3.7/site-packages/sphinx_autodoc_typehints.py", line 60, in get_annotation_args
    original = getattr(sys.modules[module], class_name)
KeyError: 'numpy'
agronholm commented 5 years ago

What is the offending annotation that fails to render?

hagenw commented 5 years ago

I got the same for pandas:

Exception occurred:
  File "/home/audeering.local/hwierstorf/.envs/audata/lib/python3.6/site-packages/sphinx_autodoc_typehints.py", line 60, in get_annotation_args
    original = getattr(sys.modules[module], class_name)
KeyError: 'pandas.api.types'

How can I figure out what the offending annotation was exactly?

agronholm commented 5 years ago

Edit the sphinx_autodoc_typehints module and place a try...except block to catch the KeyError and add either a print() call that prints the annotation or a debugger breakpoint in there.

agronholm commented 5 years ago

Alternatively, if you have an open source project, you can point me to it and I can do that for you.

hagenw commented 5 years ago

OK, I changed the beginning of get_annotation_args to:

def get_annotation_args(annotation, module: str, class_name: str) -> Tuple:
    try:
        original = getattr(sys.modules[module], class_name)
    except KeyError:
        print('annotation: {}'.format(annotation))
        print('module: {}'.format(module))
        print('class_name: {}'.format(class_name))
        raise
    except AttributeError:
        pass

And this returns:

annotation: pandas.api.types.CategoricalDtype
module: pandas.api.types
class_name: CategoricalDtype

For me it worked with version 1.9.0.

agronholm commented 5 years ago

Thanks. It strikes me as odd that the module containing this annotation was not loaded. Can you find the function signature containing this annotation and paste it for me here, along with the associated import?

hagenw commented 5 years ago

I'm not completely sure, if this is the correct one, but I have something like this:

from typing import Sequence

import pandas as pd

class Table(HeaderBase):

    def __init__(self,
                 files: Sequence[str],
                 *,
                 starts: Sequence[pd.Timedelta] = None,
                 ends: Sequence[pd.Timedelta] = None)
hagenw commented 5 years ago

Note, that in conf.py I also have something like: autodoc_mock_imports = ['pandas']

And I realized now, that it was also not build completely correct with 1.9.0: Opera Snapshot_2019-11-04_175242

hagenw commented 5 years ago

Removing autodoc_mock_imports fixes this error and it runs through with 1.10.0. The result looks like: api-table html

Which is fine, only the Timedelta entry is not a clickable link.

agronholm commented 5 years ago

@jklaise Are you also using autodoc_mock_imports?

agronholm commented 5 years ago

@hagenw Do you have Pandas installed when building the docs, and the Intersphinx inventory loaded for Pandas? If not, there is no way for Sphinx to create the correct link.

jklaise commented 5 years ago

@agronholm I am indeed using autodock_mock_imports, I can debug more tomorrow.

hagenw commented 5 years ago

@hagenw Do you have Pandas installed when building the docs, and the Intersphinx inventory loaded for Pandas? If not, there is no way for Sphinx to create the correct link.

Yes, pandas is installed and intersphinx setup correctly. I could try to setup a minimal open source project on Wednesday that tries to reproduce the behavior.

Maybe the missing link is related to https://github.com/agronholm/sphinx-autodoc-typehints/issues/47

agronholm commented 5 years ago

I could try to setup a minimal open source project on Wednesday that tries to reproduce the behavior.

Yes, that would be very helpful.

agronholm commented 5 years ago

I have been wondering if the Sphinx plugins could somehow access the objects inventory without having to load it themselves. That might make it possible to fix #47.

agronholm commented 5 years ago

I've just released v1.10.1 which fixes this issue. I also added a regression test to make sure it won't happen again.

hagenw commented 5 years ago

Thank you very much for your effort.

If I use autodock_mock_imports as before now everything runs smoothly and I get the correct links. If I run without autodock_mock_imports everything runs smoothly, only the links are missing, as mentioned in https://github.com/agronholm/sphinx-autodoc-typehints/issues/116#issuecomment-549449530