sizmailov / pybind11-stubgen

Generate stubs for python modules
Other
236 stars 47 forks source link

Enum with custom `__str__` results in non-type-checkable stub #207

Open bluenote10 opened 10 months ago

bluenote10 commented 10 months ago

The following is a minimal example involving an enum with a custom __str__ implementation:

#include <stdexcept>
#include <string>

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

enum class MyEnum
{
  FOO,
  BAR,
};

std::string customToString(MyEnum value)
{
  switch (value) {
    case MyEnum::FOO:
      return "foo";
    case MyEnum::BAR:
      return "bar";
    default:
      throw std::invalid_argument("Invalid value");
  }
}

PYBIND11_MODULE(my_native_module, m)
{
  pybind11::enum_<MyEnum>(m, "MyEnum")
      .value("FOO", MyEnum::FOO)
      .value("BAR", MyEnum::BAR)
      .def("__str__", &customToString);
}

Running the stub generator on this module produces:

from __future__ import annotations
import typing
__all__ = ['MyEnum']
class MyEnum:
    """
    Members:

      FOO

      BAR
    """
    BAR: typing.ClassVar[MyEnum]  # value = <MyEnum.BAR: 1>
    FOO: typing.ClassVar[MyEnum]  # value = <MyEnum.FOO: 0>
    __members__: typing.ClassVar[dict[str, MyEnum]]  # value = {'FOO': <MyEnum.FOO: 0>, 'BAR': <MyEnum.BAR: 1>}
    @staticmethod
    def name(*args, **kwargs):
        """
        __str__(*args, **kwargs)
        Overloaded function.

        1. __str__(self: handle) -> str

        2. __str__(self: my_native_module.MyEnum) -> str
        """
    def __eq__(self, other: typing.Any) -> bool:
        ...
    def __getstate__(self) -> int:
        ...
    def __hash__(self) -> int:
        ...
    def __index__(self) -> int:
        ...
    def __init__(self, value: int) -> None:
        ...
    def __int__(self) -> int:
        ...
    def __ne__(self, other: typing.Any) -> bool:
        ...
    def __repr__(self) -> str:
        ...
    def __setstate__(self, state: int) -> None:
        ...
    @typing.overload
    def __str__(self) -> str:
        ...
    @typing.overload
    def __str__(self) -> str:
        ...
    @property
    def value(self) -> int:
        ...

The issue here is that mypy isn't able to type check this stub because of the redundant overload, i.e., trying to use the stub with mypy errors with:

out/my_native_module.pyi:47: error: Overloaded function signature 2 will never be matched: signature 1's parameter type(s) are the same or broader  [misc]
Found 1 error in 1 file (checked 1 source file)

This issue seems to be related to https://github.com/pybind/pybind11/issues/4585 because there is also something funny happening related to name (which should be a property, but somehow adding a custom __str__ switches it to a method instead -- it is actually an instancemethod not a staticmethod though). Inspecting the underlying annotation produced by pybind11 at runtime:

In [1]: import my_native_module

In [2]: print(my_native_module.MyEnum.__str__.__doc__)
__str__(*args, **kwargs)
Overloaded function.

1. __str__(self: handle) -> str

2. __str__(self: my_native_module.MyEnum) -> str

In [3]: print(my_native_module.MyEnum.name.__doc__)
__str__(*args, **kwargs)
Overloaded function.

1. __str__(self: handle) -> str

2. __str__(self: my_native_module.MyEnum) -> str

So in a sense the root cause is the broken annotation produced by pybind11 here, but perhaps it is an easy fix on the stub generator side to omit such redundant overloads as a robustification? (The mypy stubgen just seems to ignore them, and therefore still produces type-checkable output in this case.)