search

kalekundert / autoclasstoc

Sphinx plugin to make easier-to-navigate class documentation.
MIT License
17 stars 7 forks source link

Error when module's qualified name is different from the module file location #21

Open whitews opened 2 years ago

whitews commented 2 years ago

Hi Kale,

Thanks for your work on this project, I find it very useful for documenting large classes.

I have a project where, via __init__ files, the qualified names differ from the file path for files containing some class definitions. When I try to add .. autoclasstoc:: to an existing RST doc (that successfully builds prior to adding), I get a ModuleNotFound error. For example, I have a RectangleGate class that has the qualified path flowkit.gates.RectangleGate and would be imported like:

from flowkit.gates import RectangleGate

but the actual file path to the module where the class is defined is:

flowkit/_models/gates/_gates.py

This was a working docs build prior to adding the autoclasstoc line for this class. The RST file that triggers the error:

RectangleGate Class
===================

.. currentmodule:: flowkit.gates

.. autoclass:: RectangleGate
    :members:

    .. autoclasstoc::

Full stack trace:

Traceback (most recent call last):
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/application.py", line 329, in build
    self.builder.build_update()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 288, in build_update
    self.build(to_build,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 302, in build
    updated_docnames = set(self.read())
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 409, in read
    self._read_serial(docnames)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 430, in _read_serial
    self.read_doc(docname)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 483, in read_doc
    publisher.publish()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/io.py", line 103, in read
    self.parse()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/parsers.py", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2779, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_markup
    self.explicit_list(blank_finish)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2379, in explicit_list
    newline_offset, blank_finish = self.nested_list_parse(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 318, in nested_list_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_markup
    self.explicit_list(blank_finish)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2379, in explicit_list
    newline_offset, blank_finish = self.nested_list_parse(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 318, in nested_list_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2657, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct
    return method(self, expmatch)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2101, in directive
    return self.run_directive(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive
    result = directive_instance.run()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/ext/autodoc/directive.py", line 159, in run
    result = parse_generated_content(self.state, params.result, documenter)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/ext/autodoc/directive.py", line 106, in parse_generated_content
    state.nested_parse(content, 0, node)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2352, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct
    return method(self, expmatch)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2101, in directive
    return self.run_directive(
 File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive
    result = directive_instance.run()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/domains/__init__.py", line 281, in run
    return super().run()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/sphinx/directives/__init__.py", line 199, in run
    self.state.nested_parse(self.content, self.content_offset, contentnode)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2352, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct
    return method(self, expmatch)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2101, in directive
    return self.run_directive(
  File "../envs/flowkit-dev/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive
    result = directive_instance.run()
  File "../envs/flowkit-dev/lib/python3.10/site-packages/autoclasstoc/plugin.py", line 29, in run
    mod, cls = utils.load_class(mod_name, cls_name)
  File "../envs/flowkit-dev/lib/python3.10/site-packages/autoclasstoc/utils.py", line 46, in load_class
    mod = import_module(mod_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flowkit.gates'
kalekundert commented 2 years ago

Thanks for the bug report. I'm not immediately sure what's causing the problem, since it seems like the code already trying to do the right thing: import flowkit.gates via importlib.import_module (which should behave identically to import flowkit.gates).

Can you double-check that sys.path is set correctly? I found the FlowKit github repo and took a look at conf.py. The sys.path.insert(0, os.path.abspath('../..')) approach is fragile, since it's relative to the directory that sphinx is invoked from. Better would be to (i) install flowkit using pip and not change sys.path at all or (ii) configure sys.path relative to __file__, e.g. sys.path.insert(os.path.dirname(os.path.dirname(__file__))) or similar.

Let me know if either of those approaches solve the problem, otherwise I'll look into this more over the weekend.

Completely unrelated to this issue, I have a library that I use for flow cytometry analysis that I want to shamelessly plugbring to your attention, since it seems complementary to FlowKit and might be useful to you. It's called wellmap, and it's basically a simple text file format that allows you to describe which experimental conditions are tested in which wells of a microplate. It's very useful for writing analysis scripts (such as those using FlowKit) that aren't tied to one specific plate layout.

whitews commented 2 years ago

It's a small world! I starred wellmap and will check it out.

I also questioned whether that janky sys path business could be the culprit. I just changed my local version to insert os.path.dirname(os.path.dirname(__file__)) but got the same error. Another odd thing is there doesn't seem to be any autoclasstoc code referenced in the stack trace, but maybe there are callbacks in the inner workings of sphinx.

kalekundert commented 2 years ago

I know! As soon as I saw flowkit.gates, I was thinking "that sounds like a flow cytometry library!". It looks nice, too, I'll give it a try the next time I have flow data to analyze.

Anyways, I'll have to take a closer look at this bug. The stack trace you copied above has two autoclasstoc lines right at the end, so I don't think there's anything odd in that respect.