problems using `:members:` for autoclass in v0.8.0

[2bndy5]

Latest release (v0.8.0) seems to have broken the use of the autoclass directive's :members: option. I'm getting the following unhelpful error

Exception occurred:
  File "/usr/local/lib/python3.10/inspect.py", line 2269, in _signature_from_builtin
    raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <built-in method  of PyCapsule object at 0x7f5404c39440>

The traceback isn't helpful at all. the only thing I notice from the error message is that the object's name is missing: objtype ?? at offset.

full traceback

``` Traceback (most recent call last): File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/util/inspect.py", line 576, in signature signature = inspect.signature(subject, follow_wrapped=True) File "/usr/local/lib/python3.10/inspect.py", line 3245, in signature return Signature.from_callable(obj, follow_wrapped=follow_wrapped, File "/usr/local/lib/python3.10/inspect.py", line 2993, in from_callable return _signature_from_callable(obj, sigcls=cls, File "/usr/local/lib/python3.10/inspect.py", line 2459, in _signature_from_callable return _signature_from_builtin(sigcls, obj, File "/usr/local/lib/python3.10/inspect.py", line 2269, in _signature_from_builtin raise ValueError("no signature found for builtin {!r}".format(func)) ValueError: no signature found for builtin During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/cmd/build.py", line 276, in build_main app.build(args.force_all, filenames) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/application.py", line 323, in build self.builder.build_all() File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 252, in build_all self.build(None, summary=__('all source files'), method='all') File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 302, in build updated_docnames = set(self.read()) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 409, in read self._read_serial(docnames) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 430, in _read_serial self.read_doc(docname) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 483, in read_doc publisher.publish() File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/core.py", line 217, in publish self.document = self.reader.read(self.source, self.parser, File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/io.py", line 103, in read self.parse() File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/readers/__init__.py", line 78, in parse self.parser.parse(self.input, document) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/parsers.py", line 78, in parse self.statemachine.run(inputlines, document, inliner=self.inliner) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 170, in run results = StateMachineWS.run(self, input_lines, input_offset, File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run context, next_state, result = self.check_line( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line return method(match, context, next_state) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2779, in underline self.section(title, source, style, lineno - 1, messages) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 327, in section self.new_subsection(title, lineno, messages) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection newabsoffset = self.nested_parse( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse state_machine.run(block, input_offset, memo=self.memo, File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run results = StateMachineWS.run(self, input_lines, input_offset) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run context, next_state, result = self.check_line( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line return method(match, context, next_state) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2779, in underline self.section(title, source, style, lineno - 1, messages) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 327, in section self.new_subsection(title, lineno, messages) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection newabsoffset = self.nested_parse( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse state_machine.run(block, input_offset, memo=self.memo, File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run results = StateMachineWS.run(self, input_lines, input_offset) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run context, next_state, result = self.check_line( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line return method(match, context, next_state) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2779, in underline self.section(title, source, style, lineno - 1, messages) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 327, in section self.new_subsection(title, lineno, messages) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection newabsoffset = self.nested_parse( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse state_machine.run(block, input_offset, memo=self.memo, File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run results = StateMachineWS.run(self, input_lines, input_offset) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 240, in run context, next_state, result = self.check_line( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/statemachine.py", line 452, in check_line return method(match, context, next_state) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2352, in explicit_markup nodelist, blank_finish = self.explicit_construct(match) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct return method(self, expmatch) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2101, in directive return self.run_directive( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive result = directive_instance.run() File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/ext/autodoc/directive.py", line 148, in run documenter.generate(more_content=self.content) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 1789, in generate return super().generate(more_content=more_content, File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 955, in generate self.document_members(all_members) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 1780, in document_members super().document_members(all_members) File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 831, in document_members documenter.generate( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 884, in generate if not self.import_object(): File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx_immaterial/apidoc/python/autodoc_property_type.py", line 57, in import_object signature = sphinx.util.inspect.signature( File "/mnt/c/Users/username/Documents/GitHub/env/lib/python3.10/site-packages/sphinx/util/inspect.py", line 579, in signature signature = inspect.signature(subject) File "/usr/local/lib/python3.10/inspect.py", line 3245, in signature return Signature.from_callable(obj, follow_wrapped=follow_wrapped, File "/usr/local/lib/python3.10/inspect.py", line 2993, in from_callable return _signature_from_callable(obj, sigcls=cls, File "/usr/local/lib/python3.10/inspect.py", line 2459, in _signature_from_callable return _signature_from_builtin(sigcls, obj, File "/usr/local/lib/python3.10/inspect.py", line 2269, in _signature_from_builtin raise ValueError("no signature found for builtin {!r}".format(func)) ValueError: no signature found for builtin Exception occurred: File "/usr/local/lib/python3.10/inspect.py", line 2269, in _signature_from_builtin raise ValueError("no signature found for builtin {!r}".format(func)) ValueError: no signature found for builtin ```

I am using pybind11 with custom signatures (not the pybind11 auto-generated ones) to build the python binding from C++.

simple example

```cpp .def(py::init(), R"docstr( __init__(ce_pin: int, csn_pin: int, spi_speed: int = 10000000) __init__(spi_speed: int = 10000000) Create a RF24 object. :param int ce_pin: The pin number connected to the radio's CE pin. :param int csn_pin: The pin number connected to the radio's CSN pin. :param int spi_speed: The SPI bus speed (in Hz). Defaults to 10 MHz when not specified. )docstr", py::arg("ce_pin") = 0xFFFF, py::arg("csn_pin") = 0xFFFF, py::arg("spi_speed") = 10000000) .def(py::init(), R"docstr( If it is desirable to create a RF24 object in which the pin numbers are dynamically configured, the ``ce_pin`` and ``csn_pin`` parameters can be omitted. )docstr", py::arg("spi_speed") = 10000000) ``` Note: The separating backslash `\` isn't required in sphinx v4.0+ In v0.7.3, this renders as 

At first I, thought it was related to my overloaded __init__(), but it seems to be happening anytime I specify the members option.

As a workaround I can use

.. autoclass:: pyrf24.rf24.RF24

    .. automethod:: __init__
    .. ... keep listing all the functions/attributes this way

but it breaks when I use

.. autoclass:: pyrf24.rf24.RF24
    :members: __init__

.. autoclass:: pyrf24.rf24_mesh.AddrListStruct
    :members:

    .. this class only has a c'tor and 2 instance attributes

[jbms]

The issue appears to be with autodoc_property_type.py

This has caused issues before --- it would probably be nice if we had a unit test to check for issues and prevent regressions. I think using pybind11 in the build process will be more trouble than it is worth, but perhaps we can simulate it.

[2bndy5]

I was thinking of building docs for one of the pybind11 example repos, but that may also catch bugs in pybind11.

Currently pybind11 doesn't append provided docstrings for class attributes...

[jbms]

Can you perhaps create a self-contained example as a test case?

[2bndy5]

In trying to create a simple example project, I think I narrowed it down to class instance attributes. Although, I'm starting to get the feeling that this may have something to do with the builtin "magic" methods that pybind11 automatically adds...

The pybind11-doc-test project resembles the original repo in which I discovered this issue (just super simplified).

• Both methods (__init__() and add()) of the class are overloaded to check it had nothing to do with using custom signatures embedded in the docstrings.
• The class has 2 bool attributes: ¹read-only and ²dynamic.

The sample's docs are designed to show the error (currently broken).

The docs can be generated by removing the attributes' names from the :members: option.

```diff .. autoclass:: sphinx_immaterial_pybind11_doc_test.example.CppMath - :members: __init__, add, is_set_by_init, dynamic_attribute + :members: __init__, add - .. .. autoattribute:: is_set_by_init - .. .. autoattribute:: dynamic_attribute + .. autoattribute:: is_set_by_init + .. autoattribute:: dynamic_attribute ```

I plan on uploading this sample project to test.pypi, so we can run pytest with it in this repo's CI.

[2bndy5]

oh yea, check the doc-test project's CI artifacts for the built wheels (currently only targeting the OSs used in github runners)

[jbms]

Thanks. I think potentially it would make sense to just include the main.cpp and a minimal setup.py-based build as a test within the sphinx-immaterial repo --- using setup.py would probably be simple than relying on cmake.

[2bndy5]

How do want to rope in the pybind11 source? In the sample project, I used CMake. In the the pyrf24 project, I used a git submodule.

[jbms]

It appears to be available on pypi.

[2bndy5]

took me a while to figure out how to do it without cmake, but see #145

[2bndy5]

been looking into the autodoc_proprerty.py, and I found an unused var (fget): https://github.com/jbms/sphinx-immaterial/blob/412922e592318f261d7d1e965e9716f69afefa98/sphinx_immaterial/apidoc/python/autodoc_property_type.py#L28-L40

fget isn't actually used. However, this issue wasn't resolved when I changed

-    doc = obj.__doc__
+    doc = fget.__doc__

---

The working solution I did find was to make https://github.com/jbms/sphinx-immaterial/blob/412922e592318f261d7d1e965e9716f69afefa98/sphinx_immaterial/apidoc/python/autodoc_property_type.py#L19-L25

only return sphinx.util.inspect.safe_getattr(obj, "func", None)

[2bndy5]

I found a peculiarity in pybind11 concerning a class' property's fget.__doc__.

If I build the test pkg using a singular module structure:

- sphinx-immaterial-pybind11-issue-134
    - setup.py
    - sphinx_immaterial_pybind11_issue_134.cpp

then Example.is_set_by_init.fget.__doc__ is as expected:

(self: sphinx_immaterial_pybind11_issue_134.Example) -> bool

results from rich.inspect()

``` >>> from rich import inspect >>> from sphinx_immaterial_pybind11_issue_134 import Example >>> inspect(Example.is_set_by_init, all=True) ╭───────────────────────────────── ──────────────────────────────────╮ │ This read-only ``bool`` attribute is set by the constructor. │ │ │ │ ╭───────────────────────────────────────────────────────────────────────────────────╮ │ │ │ │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ __doc__ = '\n This read-only ``bool`` attribute is set by the │ │ constructor.\n ' │ │ fdel = None │ │ fset = None │ │ __isabstractmethod__ = False │ │ __class__ = class __class__(fget=None, fset=None, fdel=None, doc=None): │ │ Property attribute. │ │ __delattr__ = def __delattr__(name, /): Implement delattr(self, name). │ │ __delete__ = def __delete__(instance, /): Delete an attribute of instance. │ │ deleter = def deleter(...) Descriptor to obtain a copy of the property │ │ with a different deleter. │ │ __dir__ = def __dir__(): Default dir() implementation. │ │ __eq__ = def __eq__(value, /): Return self==value. │ │ fget = def fget(...) (self: │ │ sphinx_immaterial_pybind11_issue_134.Example) -> bool │ │ __format__ = def __format__(format_spec, /): Default object formatter. │ │ __ge__ = def __ge__(value, /): Return self>=value. │ │ __get__ = def __get__(instance, owner=None, /): Return an attribute of │ │ instance, which is of type owner. │ │ __getattribute__ = def __getattribute__(name, /): Return getattr(self, name). │ │ getter = def getter(...) Descriptor to obtain a copy of the property │ │ with a different getter. │ │ __gt__ = def __gt__(value, /): Return self>value. │ │ __hash__ = def __hash__(): Return hash(self). │ │ __init__ = def __init__(*args, **kwargs): Initialize self. See │ │ help(type(self)) for accurate signature. │ │ __init_subclass__ = def __init_subclass__(...) This method is called when a class │ │ is subclassed. │ │ __le__ = def __le__(value, /): Return self<=value. │ │ __lt__ = def __lt__(value, /): Return self

But, if I build the test pkg using a pkg src folder:

- sphinx-immaterial-pybind11-issue-134
    - setup.py
    - src
        - sphinx_immaterial_pybind11_issue_134.cpp

Then Example.is_set_by_init.fget.__doc__ is not set at all.

return_annotation = None in the Signature object used to parse the property in sphinx_immaterial/apigen/python/autodoc_property_type.py

results from rich.inspect()

``` >>> from rich import inspect >>> from sphinx_immaterial_pybind11_issue_134 import Example >>> inspect(Example.is_set_by_init, all=True) ╭─────────────────────────────────── ───────────────────────────────────╮ │ This read-only ``bool`` attribute is set by the constructor. │ │ │ │ ╭──────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ __doc__ = '\n This read-only ``bool`` attribute is set by the │ │ constructor.\n ' │ │ fdel = None │ │ fset = None │ │ __isabstractmethod__ = False │ │ __class__ = class __class__(fget=None, fset=None, fdel=None, doc=None): │ │ Property attribute. │ │ __delattr__ = def __delattr__(name, /): Implement delattr(self, name). │ │ __delete__ = def __delete__(instance, /): Delete an attribute of instance. │ │ deleter = def deleter(...) Descriptor to obtain a copy of the property with │ │ a different deleter. │ │ __dir__ = def __dir__(): Default dir() implementation. │ │ __eq__ = def __eq__(value, /): Return self==value. │ │ fget = def fget(...) │ │ __format__ = def __format__(format_spec, /): Default object formatter. │ │ __ge__ = def __ge__(value, /): Return self>=value. │ │ __get__ = def __get__(instance, owner=None, /): Return an attribute of │ │ instance, which is of type owner. │ │ __getattribute__ = def __getattribute__(name, /): Return getattr(self, name). │ │ getter = def getter(...) Descriptor to obtain a copy of the property with │ │ a different getter. │ │ __gt__ = def __gt__(value, /): Return self>value. │ │ __hash__ = def __hash__(): Return hash(self). │ │ __init__ = def __init__(*args, **kwargs): Initialize self. See │ │ help(type(self)) for accurate signature. │ │ __init_subclass__ = def __init_subclass__(...) This method is called when a class is │ │ subclassed. │ │ __le__ = def __le__(value, /): Return self<=value. │ │ __lt__ = def __lt__(value, /): Return self

---

So, I don't think the return type annotation for class properties can be fully supported in pybind11 wrapped pkgs. Technically, I think this may be a bug in pybind11, but that's not our concern here. We just want the property's basic signature at least.

[jbms]

I'm a bit confused --- for me, regardless of whether the C++ source file in a src/ directory or not, I'm seeing: sphinx_immaterial_pybind11_issue_134.Example.is_set_by_init.fget.__doc__ equal to None

[2bndy5]

FYI, I just did a recently release of the problematic repo, pyrf24, where I moved the binding code out of the repo's src/pkg_name/ folder (its now just in src/) in an effort to discontinue using scikit-build pkg. I'm not sure how this may have effected the issue here...

I say this because I because (using the latest release of pyrf24) the members option seems to work with your fix from #142 . I also get expected results from my suggested fix in https://github.com/jbms/sphinx-immaterial/pull/142#issuecomment-1216481744. I'm not sure which solution you want to use; I leave that up to you.

As it stands I think #145 is an accurate production of this issue because it does fail without the fix in #142 . I'll mark it ready to merge, so I can move on from this problem.

I'm not concerned with getting the docstring from c-extended class properties' fget() because it is a pybind11 problem. Maybe we can revisit that edge case later, but I want to get back to more important things (like finishing the cpp-apigen ext).

[2bndy5]

fixed in v0.9.0