autodoc confused by angle brackets ("<" or ">")

[MKuranowski]

Describe the bug

autodoc creates very verbose signatures for functions and methods, as if the tilde (~) sign was ignored, whenever there's a default value with an angle bracket (< or >) in its representation.

How to Reproduce

Default configuration (as generated by sphinx-quickstart), but with the sphinx.ext.autodoc extension.

index.rst:

mymodule documentation
======================

.. automodule:: mymodule
   :members:
   :undoc-members:

mymodule.py:

def identity[T](x: T) -> T:
    return x

def foo(a: Callable[[int], int] = identity) -> int:
    """foo evaluates ``a`` at zero."""
    return a(0)

With that configuration, the generated summary looks like this:

mymodule.foo(a: ~typing.Callable[[int], int] = <function identity>) → int

    foo evaluates a at zero.

---

Without the default argument value for foo (def foo(a: Callable[[int], int]) -> int:, the generated summary is correct and looks like this:

mymodule.foo(a: Callable[[int], int]) → int

    foo evaluates a at zero.

Environment Information

Platform:              linux; (Linux-6.10.1-arch1-1-x86_64-with-glibc2.40)
Python version:        3.12.4 (main, Jun  7 2024, 06:33:07) [GCC 14.1.1 20240522])
Python implementation: CPython
Sphinx version:        7.4.7
Docutils version:      0.21.2
Jinja2 version:        3.1.4
Pygments version:      2.18.0

Sphinx extensions

["sphinx.ext.autodoc"]

Additional context

No response

[electric-coder]

Default configuration

The deciding configuration here is autodoc's autodoc_preserve_defaults (notice the docs still say: "Added in version 4.0: Added as an experimental feature." although in my experience this functionality has been stable). It tries to preserve the default argument values in callables to include them in the signature's documentation.

def foo(a: Callable[[int], int] = identity) -> int:

Here the default value is identity, the name of a callable function. autodoc should preserve default argument values textually "as is" and that's the desired behavior.

mymodule.foo(a: ~typing.Callable[[int], int] = <function identity>) → int

Here are 2 strange things:

1. The tilde ~ shouldn't be included in the rendered docs, when it's included in the .rst or the appropriate docstring fields the result should be to show a shortened signature instead of the fully qualified name.
2. The text <function identity> seems like a runtime substitution of identity.

So yes, I haven't tested it but if the results are as shown this is a bug. It's especially strange because default values are rarely touched and autodoc just transcribes the values textually as strings.

P.S. The title of the question is misleading - if I understand the example correctly- because the default argument value you're trying to document doesn't have any greater/lesser < or > signs. It's just the name of a callable function that apparently autodoc substitutes for a runtime representation, and it's that runtime representation of the callable that's written between < >.

With that configuration, the generated summary looks like this:

A screenshot would have been valuable to illustrate such strange symptoms.

[electric-coder]

There's a similar problem reported with square brackets [ ] #9704, in general this kind of bug was fixed by #8771.

[MKuranowski]

A screenshot would have been valuable to illustrate such strange symptoms.

[...] default argument value you're trying to document doesn't have any greater/lesser < or > signs

repr(identity) == "<function identity>"

Sorry, this was an omission on my side, I have seen this with other objects (not just functions, predominantly dataclass factories) and all thy had in common was the representation containing < and >. Take a look at this example:

mymodule.py:

from dataclasses import dataclass
from typing import Any

@dataclass
class WithRepr:
    repr: str

    def __repr__(self) -> str:
        return self.repr

def foo1(x: Any = WithRepr("<angle brackets>")) -> None:
    pass

def foo2(x: Any = WithRepr("<")) -> None:
    pass

def foo3(x: Any = WithRepr(">")) -> None:
    pass

Results in:

[electric-coder]

As was said by tk0miya (emphasis mine):

• It preserves the default argument values of function signatures in source code and keep them not evaluated for readability.

and the documentation also states this:

autodoc_preserve_defaults

If True, the default argument values of functions will be not evaluated on generating document. It preserves them as is in the source code.

Presently the most viable workaround is using the exclude-members option and pasting affected signatures into their own autodoc directives in the .rst file, something like:

.. automodule:: mymodule
   :members:
   :undoc-members:
   :exclude-members: foo

   .. autofunction:: foo(a: typing.Callable[[int], int] = identity) -> int

or for the shortened type using the tilde ~:

.. automodule:: mymodule
   :members:
   :undoc-members:
   :exclude-members: foo

   .. autofunction:: foo(a: ~typing.Callable[[int], int] = identity) -> int

This is certain to not be a good solution for autosummary users who seek to write less .rst and for other users it's also an annoyance.

[MKuranowski]

This workaround still doesn't work with < or >.

index.rst:

mymodule documentation
======================

.. automodule:: mymodule

    .. autofunction:: foo(a: ~typing.Callable[[int], int] = <function identity>) -> int

mymodule.py:

from typing import Callable

def identity[T](x: T) -> T:
    return x

def foo(a: Callable[[int], int] = identity) -> int:
    """foo evaluates ``a`` at zero."""
    return a(0)

Result:

[electric-coder]

@MKuranowski you didn't remove the greater > lesser < signs from the .rst, neither did you exclude the function in the module. Carefully look at the example I included.

[MKuranowski]

The explicit signature from the .rst takes precedence, at least from what I have observed. And yes, removing < helps, but that's a solution akin to "My car's engine makes a weird noise" "Don't turn on your car then".

There's something deeply wrong with processing angle brackets. I can't find any special meaning of < and > in the reStructuredText spec outside of interpreted text, hyperlinks and option lists; and I don't see any of the rules applying here.

[electric-coder]

but that's a solution akin (...)

That's not a good analogy because it's a solution that makes things work! Having one single clear solution (explicitly writing the signature into the rst) is good enough.

and I don't see any of the rules applying here.

Presumably the directive argument is being parsed incorrectly, look no further if you want to assign blame for the difficulty of writing a type hint compatible signature parser for an ever changing specification. As the saying goes in the open source community: "be the change you want to see"; iow: "PRs welcome".

You could also write a signature as previously explained, or use a sentinel value instead of a mutable default argument (as is generally recommended).

[electric-coder]

I can't find any special meaning of < and > in the reStructuredText spec outside of (...)

Here it gets really complicated! Because when Sphinx is translating to HTML the angle brackets do have a special meaning and can break the parsing. A directive is a body element and there's not necessarily any limitation on the directive argument, if you look carefully at the docs for directive it says:

Directives

Doctree elements:

depend on the directive

That gives custom directives plenty of latitude for how they are parsed, hence for autoclass, autofunction the directive argument must be able to accommodate any Python signature and produce correct output. The solution for the problem shouldn't need escape mechanisms much less require users to read the docutils DTD. So it's a bug...

[MKuranowski]

[...] makes things work!

Well, not with < or >; and that's what the bug is about :wink:.

I think that in the end I will move away from autosummary and just use autodoc, not only due to this bug. This is however really painful for maintenance, as there is no single source for signatures, and programmers need to remember to update the documentation with any changes to code.