Closed Yikai-Liao closed 10 months ago
The annotation comes from your code, there is no issue on pybind11-stubgen
side.
The annotation comes from your code, there is no issue on
pybind11-stubgen
side.
So how can I get a right annotation by changing my code? I'm just confused about it.
@Yikai-Liao I am seeing this on the project I am working on also. The only dependencies we have in common are pybind11 and eigen.
I see the same type of expression here: libigl-python-bindings, which contains a bunch of eigen code. I'm still not exactly sure how they are getting there though.
The annotation is generated by pybind11
.
pybind11-stubgen
has two built-in options to deal with that, see --help
for a bit more details.
pybind11-stubgen [-h]
[-o OUTPUT_DIR]
[--root-suffix ROOT_SUFFIX]
[--ignore-invalid-expressions REGEX]
[--ignore-invalid-identifiers REGEX]
[--ignore-unresolved-names REGEX]
[--ignore-all-errors]
[--enum-class-locations REGEX:LOC]
[--numpy-array-wrap-with-annotated| # <-- first
--numpy-array-remove-parameters] # <-- second
[--print-invalid-expressions-as-is]
[--print-safe-value-reprs REGEX]
[--exit-code]
[--stub-extension EXT]
MODULE_NAME
In its type stubs, numpy.ndarray
is generic w.r.t. 2 type variables:
from __future__ import annotations
from typing import Any, Generic, TypeVar
import numpy as np
_DType_co = TypeVar("_DType_co", covariant=True, bound=np.dtype[Any])
_ShapeType = TypeVar("_ShapeType", bound=Any)
class ndarray(Generic[_ShapeType, _DType_co]):
...
And you can utilize them like this:
from __future__ import annotations
from typing import TypeVar
import numpy as np
from typing_extensions import Literal
M = TypeVar("M")
N = TypeVar("N")
P = TypeVar("P")
def matmul(
l: np.ndarray[tuple[M, N], np.dtype[np.float64]],
r: np.ndarray[tuple[N, P], np.dtype[np.float64]],
) -> np.ndarray[tuple[M, P], np.dtype[np.float64]]:
return l @ r
arr1: np.ndarray[tuple[Literal[2], Literal[3]], np.dtype[np.float64]] = np.array([[1, 2, 3], [4, 5, 6]], dtype=np.float64)
arr2: np.ndarray[tuple[Literal[3], Literal[4]], np.dtype[np.float64]] = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], dtype=np.float64)
arr3 = matmul(arr1, arr2) # pyright reveals the type is: ndarray[tuple[Literal[2], Literal[4]], dtype[float64]]
arr3.shape # (2, 4)
What would you think about generating this style of annotations? You can see that it works, and it seems like this is what people are currently doing in numpy:
https://github.com/numpy/numpy/issues/16544#issuecomment-1470501380 https://github.com/numpy/numpy/issues/16544#issuecomment-1804251364
The pybind11
style:
numpy.ndarray[numpy.float32[m, 1]]
would become:
M = typing.TypeVar("M")
... # any others
numpy.ndarray[tuple[M, typing_extensions.Literal[1]], numpy.dtype[numpy.float32]]
The --numpy-array-wrap-with-annotated
currently produces the following stub
https://github.com/sizmailov/pybind11-stubgen/blob/afc508f5709b9e08ebdd5deb01021d3d23f9d16d/tests/stubs/python-3.12/pybind11-master/demo/_bindings/eigen.pyi#L35-L38
I think it's good enough.
If you are interested in adding different flavors of annotating numpy arrays, take a look at #113 and #115.
The return type in c++ is define as the Pianoroll below:
pybind11-stubgen just generate this strange type hint for my project:
numpy.ndarray[bool[128, n]]
And this kind of type is wrong in python.
And here is my project based on pybind11: pyscore