Type stubs for Python typing

[thisiscam]

Hi,

I would like to generate type stubs so that we can use type checking with islpy functions.

I'm thinking of extending gen_wrap to achieve this, but was also wondering if the maintainer has any good suggestions?

Also, it seems like the sphinx docs can already harvest the type signatures (from the generated c++ wrapper code?).

[inducer]

I think the most reasonable thing to do would be to implement this via pybind, cf. https://github.com/pybind/pybind11/issues/945. Pybind has all the information it needs, it just needs to make it visible to the interpreter. Even better, this would allow us to just drop the documentation-focused signature gathering code in gen_wrap.

[thisiscam]

@inducer Thanks for the input! Yes I agree that fixing it at the pybind level would be very desirable. However, I looked into pybind a bit and the level of unfamiliarity scared me.

Since I would really like to get the type checking to work ASAP, I did a quick implementation for typestub generation in gen_wrap: https://github.com/thisiscam/islpy/commit/e14d71acd10d628097a8d07a916be07ce4c9a696.

I'm a bit stuck on how islpy is merging the .inc files into the final wrapper?
Currently, I have to manually concatenating the files then paste them next to islpy/__init__.py.

That being said, I was able to get pretty good result and my IDE can finally auto-complete islpy functions.

_isl.pyi.txt

[inducer]

I'm a bit stuck on how islpy is merging the .inc files into the final wrapper?

They just get included, e.g. here:

https://github.com/inducer/islpy/blob/33cdc99021b1eed9be0174911202601d84ebff82/src/wrapper/wrap_isl_part3.cpp#L5

I did a quick implementation for typestub generation in gen_wrap:

I could probably be talked into merging this, if you think it's of sufficient quality. (It looks like it's got some debug leftovers in it currently, such as a breakpoint() in an exception handler.

Unfortunately, it looks like these stubs won't buy us much from the documentation angle, at least for now: https://github.com/sphinx-doc/sphinx/issues/7630.

[inducer]

Alternatively, is it possible to set .__annotations__ on the methods from the wrapper? (I haven't tried, but cls.attr("methodname").attr("__annotations__") = ... might work?)

[thisiscam]

I haven't tried, but cls.attr("methodname").attr("annotations") = ... might work?

I'm not really familiar with these annotation magics. My intuition is that one would need to pass annotations as PyObjects here (instead of annotation strings)?

[inducer]

pybind has py::object that wraps and makes C++ handling of PyObjects easier, e.g. py::make_tuple("a", "b", "c").