search

sphinx-doc / sphinx

The Sphinx documentation generator
https://www.sphinx-doc.org/
Other
6.45k stars 2.1k forks source link

Adding `multipledispatch` to Sphinx's `autodoc`. #10613

Open askrix opened 2 years ago

askrix commented 2 years ago

Is your feature request related to a problem? Please describe. In my project, I am relying on multipledispatch library to allow for multiple dispatching of functions. Following the closed feature request for autodoc to support single dispatch functions and methods, I called upon find . -type f -exec grep -F 'singledispatch' /def/null {} + in the console inside the root folder. Thus, I got the exact places to add my code snippets:

  1. sphinx/util/inspect.py, lines 281 and 292
  2. sphinx/ext/autodoc/__init__.py, lines 1314 and 2201

Having added the analogous code snippets for multipledispatch, I recompiled Sphinx, but got the error message of 

Extension error (sphinx.ext.napoleon):
Handler <function _process_docstring at 0x1060d7ee0> for event 'autodoc-process-docstring' threw an exception (exception: pop from an empty deque)
make: *** [html] Error 2

Attached, you will find the changed files. Using the command with the additional option, make html SPHINXOPTS=-v, I successfully extract the full trace back:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/sphinx/events.py", line 94, in emit
    results.append(listener.handler(self.app, *args))
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/napoleon/__init__.py", line 385, in _process_docstring
    docstring = NumpyDocstring(result_lines, app.config, app, what, name,
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/napoleon/docstring.py", line 1147, in __init__
    super().__init__(docstring, config, app, what, name, obj, options)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/napoleon/docstring.py", line 214, in __init__
    self._parse()
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/napoleon/docstring.py", line 594, in _parse
    res = self._parse_attribute_docstring()
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/napoleon/docstring.py", line 626, in _parse_attribute_docstring
    _type, _desc = self._consume_inline_attribute()
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/napoleon/docstring.py", line 303, in _consume_inline_attribute
    line = self._lines.popleft()
IndexError: pop from an empty deque

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "/usr/local/lib/python3.9/site-packages/sphinx/application.py", line 347, in build
    self.builder.build_update()
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 300, in build_update
    self.build(to_build,
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 314, in build
    updated_docnames = set(self.read())
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 421, in read
    self._read_serial(docnames)
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 442, in _read_serial
    self.read_doc(docname)
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 495, in read_doc
    publisher.publish()
  File "/usr/local/lib/python3.9/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/local/lib/python3.9/site-packages/sphinx/io.py", line 103, in read
    self.parse()
  File "/usr/local/lib/python3.9/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/usr/local/lib/python3.9/site-packages/sphinx/parsers.py", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2779, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2779, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2352, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct
    return method(self, expmatch)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2101, in directive
    return self.run_directive(
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive
    result = directive_instance.run()
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/directive.py", line 148, in run
    documenter.generate(more_content=self.content)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 955, in generate
    self.document_members(all_members)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 831, in document_members
    documenter.generate(
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 1802, in generate
    return super().generate(more_content=more_content,
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 955, in generate
    self.document_members(all_members)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 1793, in document_members
    super().document_members(all_members)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 831, in document_members
    documenter.generate(
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 952, in generate
    self.add_content(more_content)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 603, in add_content
    for i, line in enumerate(self.process_doc(docstrings)):
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 548, in process_doc
    self.env.app.emit('autodoc-process-docstring',
  File "/usr/local/lib/python3.9/site-packages/sphinx/application.py", line 458, in emit
    return self.events.emit(event, *args, allowed_exceptions=allowed_exceptions)
  File "/usr/local/lib/python3.9/site-packages/sphinx/events.py", line 102, in emit
    raise ExtensionError(__("Handler %r for event %r threw an exception") %
sphinx.errors.ExtensionError: Handler <function _process_docstring at 0x1041f0f70> for event 'autodoc-process-docstring' threw an exception (exception: pop from an empty deque)

Extension error (sphinx.ext.napoleon):
Handler <function _process_docstring at 0x1041f0f70> for event 'autodoc-process-docstring' threw an exception (exception: pop from an empty deque)
make: *** [html] Error 2

Please let me know what I am missing to make it work. Currently, I have no idea what the cause of the error might be.

Thanks in advance and have yourself an epic day! Sphinx_multidispatch.zip

AA-Turner commented 2 years ago

ping @anntzer as this might be related to the new stack impl for napolean?

In the general case, I've never heard of multipledispatch before and I don't know what it does -- the code for autodoc is already a behemoth and incredibly complicated, and I worry that adding support for what seems a little-used library / design pattern would add to that complexity.

Please do submit a PR that passes tests though, from what you've said it may be a minor change.

A

anntzer commented 2 years ago

I can have a look, but that would require a minimal repro example (a module to be autodoc'ed and a sphinx project).

AA-Turner commented 2 years ago

Agree. @askrix perhaps we can distinguish the feature request and bug report -- please could you open a (failing) PR with a patch against autodoc and a mininal reproducer?

Please then cross-link it in this issue.

A

tk0miya commented 2 years ago

Sphinx already provides app.add_autodocumenter() API to enhance autodoc from extension. So it's better to create your own extension to support multipledispatch.

binh-vu commented 2 years ago

I also have the same error message when sphinx is upgrade from 5.0.2 to 5.1.0