search

sizmailov / pybind11-stubgen

Generate stubs for python modules
Other
232 stars 47 forks source link

Add option to rewrite `numpy.ndarray` as `nptyping.NDArray` #113

Open sizmailov opened 1 year ago

sizmailov commented 1 year ago

Originated from #109 by @TheTripleV

auscompgeek commented 1 year ago

should really use numpy.typing, specifically numpy.typing.NDArray

TheTripleV commented 1 year ago

Iirc numpy.typing leaves the size unspecified while ndtyping lets you type hint/(check?) the desired size.

auscompgeek commented 1 year ago

oh, hmm. open numpy issue by the looks of things: https://github.com/numpy/numpy/issues/16544

sizmailov commented 1 year ago

The question is, who would be the consumer of the resulting stubs? The potential candidates are a programmer and a static analysis tool (IDE included).

For a human being, the shorter notation is preferable, so original pybind11 rendering is OK (more or less) IDE and static analysis tools do not currently respect the dimensions but are strict on the format of the annotation.

It would be nearly impossible to make everyone happy at once. Therefore, there would be various "flavors" of stubs of the same library for different consumers until we have a universal standard.

The nptyping seems to implement runtime checks only.

I'm sure the runtime tools (nptyping included) never use stub *.pyi files directly. So, the consumer of stubs is still a programmer, but he can be spared from the burden of transformation from one notation to another.

I included the rewrite of np.ndarray as Annotated[np.ndarray, TYPE, FixedSize(DIMS) ] (enabled by --numpy-array-wrap-with-annotated-fixed-size option) as one that 1) relatively short 2) validates with mypy 3) preserves dimension information.

Another included rewrite removes all "generic" parameters (enabled by --numpy-array-remove-parameters). It does not preserve the type or dimension, but it's trivial to implement and allows it to pass static analysis checks quickly.

We can add more rewrites (enabled by separate mutually exclusive options) to accommodate a broader range of use cases.

I like numpy.typing.NDArray more since it's in the official package, but the shape annotation must be a(!) type(!) itself, the type required to be wrapped in numpy.dtype. So it's verbose and arbitrary (for the shape part)

The nptyping is not that vague, but probably it would not be in the mainstream.

I don't see much of a problem with having both of them.

TheTripleV commented 1 year ago

The problem with Annotated for me is that the Vscode python extension doesn't show the information in annotated. I expect the majority of my users to be in vscode with Microsoft's python extension enabled.