stubgen: Class docstring not included in stub when combining `--include-docstrings` with `--inspect-mode`

[hoechenberger]

Bug Report

I'm using stubgen to generate stubs for an untyped package. Because the docstrings are assembled at runtime, I want to include them in the stubs such that IDEs using static type checkers can use them. I therefore invoke stubgen with the --include-docstrings and --inspect-mode parameters. The generated stubs do not include docstrings for classes (i.e., MyClass.__doc__), while docstrings for methods and properties are correctly added to the stubs.

If I remove the --inspect-mode switch, the class docstrings are added to the stubs as expected. But this isn't a viable solution for me, as I need to use inspect mode to generate the full docstrings.

To summarize, there seems to be an interaction between --include-docstrings and --inspect-mode, which causes class docstrings in MyClass.__doc__ to get lost.

To Reproduce

# mwe.py

class MyClass:
    """My test class."""

    def __init__(self):
        pass

    def f(self):
        """My test function."""
        pass

Run:

stubgen --include-docstrings --inspect-mode ./mwe.py

Expected Behavior

The type stub contains the class docstring.

class MyClass:
    """My test class."""
    def __init__(self) -> None: ...
    def f(self) -> None:
        """My test function."""

Actual Behavior

The type stub does not contain the class docstring.

class MyClass:
    def __init__(self) -> None: ...
    def f(self):
        """My test function."""

The expected stub can be produced by omitting the --inspect-mode parameter, i.e., by invoking:

stubgen --include-docstrings ./mwe.py

Your Environment

• Mypy version used: 1.7.0
• stubgen command-line flags: --include-docstrings --inspect-mode
• Python version used: 3.12.0

[chdominguez]

I have noticed that --include-docstrings does not include the docstrings of class properties. Is this a related issue?

[jcapriot]

Came across this issue as well, it looks like the stub generator never attempts to add the docstring for the class. https://github.com/python/mypy/blob/fe15ee69b9225f808f8ed735671b73c31ae1bed8/mypy/stubgenc.py#L860-L885

The first line written as part of the class should likely be along the lines of (similar to FunctionSig.format_sig):

        bases = self.get_base_types(cls)
        if bases:
            bases_str = "(%s)" % ", ".join(bases)
        else:
            bases_str = ""
        docstring = class_info.docstring if self._include_docstring else None
        if types or static_properties or rw_properties or methods or ro_properties or docstring:
           output.append(f"{self._indent}class {class_name}{bases_str}:")
            if docstring:
                output.append(f"\n{self._indent}    {mypy.util.quote_docstring(docstring)}")
            for line in types:
                if (
                    output
                    and output[-1]
                    and not output[-1].strip().startswith("class")
                    and line.strip().startswith("class")
                ):
                    output.append("")
                output.append(line)
            for line in static_properties:
                output.append(line)
            for line in rw_properties:
                output.append(line)
            for line in methods:
                output.append(line)
            for line in ro_properties:
                output.append(line)
        else:
            output.append(f"{self._indent}class {class_name}{bases_str}: ...")

[Josverl]

I have been working around this limitation by using libcst to do fix-ups.

I think part of the underlying question is if Docstrings should or should not be part of stub files.

In my view it is needed if there is no access to python source files, which in my ecosystem is quite common.

[mhier]

I think part of the underlying question is if Docstrings should or should not be part of stub files.

I am using stubgen also for modules written in C++, to have autocomplete and documentation available in my IDE. That does not work nicely without docstrings in the stub...

[Josverl]

Agree, my answer to that question is also: Yes they should.

But historically it has been/ was a goal to keep stubs small.

I'm inclined to think that Docstrings don't bring much processing overhead to type checkers, but that is only an assumption