search

AcademySoftwareFoundation / MaterialX

MaterialX is an open standard for the exchange of rich material and look-development content across applications and renderers.
http://www.materialx.org/
Apache License 2.0
1.88k stars 356 forks source link

Auto-Generated Stubs for Python Bindings #2024

Open joaovbs96 opened 2 months ago

joaovbs96 commented 2 months ago

Following discussion on Slack, it would be interesting to have auto-generated Stubs for the Python bindings. I would be interest in tackling this issue during devdays this year.

For context, Stub files can be used to add type hints to packages when it isn't interesting or possible to add them inline. The purpose of type hinting is, among other things, to enable type checking and code completion.

There are tools that generate stubs automatically, such as MyPy's stubgen. If we go down the path of auto-generating the stub files, they could, for example, be created during the GitHub Actions build process, storing the results as an artifact for developers to download, and even be included in the PyPi distributions.

Automatically generated stubs are often, however, not able to cover 100% of the type hints of a package and are instead more of a "template" - the remaining types would need to be "filled in" by hand. From experience, a lot of types already seem to be picked up correctly when running stubgen on MaterialX, but I can't currently estimate what or how much is missing.

I suppose one possibility would also be, instead, to commit the auto-generated stubfiles and fill the remaining bits by hand, but that would involve this extra manual step whenever API changes occur. Otherwise, we could investigate if there are ways to coerce stubgen to pick up more/all types correctly.

jstone-lucasfilm commented 2 months ago

This sounds like a good project to me, @joaovbs96, and I'd definitely recommend the auto-generated approach you suggest above, where generated Python stubs automatically remain in sync with the evolving MaterialX Python API.

joaovbs96 commented 2 months ago

I saw that OpenAssetIO generates its own stubs automatically during build time, triggered by CMake apparently. I think a similar approach could be doable here. Something I noticed is that they use pybind11-stubgen instead of MyPy's stubgen. I haven't really used pybind11-stubgen before, but I might give it a try to see how the results compare against MyPy's.

https://github.com/OpenAssetIO/OpenAssetIO/blob/2c9542230c906b9fc6d104f166c2e9a883ba4c5f/src/openassetio-python/cmodule/CMakeLists.txt#L146