WIP: Fix stubgen - Githubissues

cielavenir commented 1 year ago

@felixvd @gywn

Background

We would like to get type stub for openravepy. However we have some issues.

For example, on production HEAD, GetManipulator is like this:

In [1]: openravepy.Robot.GetManipulator?
Docstring:      
GetManipulator(self: openravepy._openravepy_0_114.openravepy_int.Robot, manipname: str) -> openravepy::PyRobotBase::PyManipulator

With this patch, GetManipulator is like this:

In [1]: openravepy.Robot.GetManipulator?
Docstring:      
GetManipulator(self: openravepy._openravepy_0_114.openravepy_int.Robot, manipname: str) -> openravepy._openravepy_0_114.openravepy_int.Manipulator

You can see the difference in return type. And latter form is required for pybind11-stubgen. This is achieved by declaring Manipulator class before Robot.GetManipulator method declaration in pybind11 entrypoint.

Result

I have fixed most order, but some cannot be fixed (yet).

ConfigurationSpecification.ExtractAffineValues has KinBody parameter. KinBody::GetConfigurationSpecification has ConfigurationSpecification parameter. But they are defined in different file.
Controller.Init has Robot parameter. Robot::GetController has Controller parameter. But they are defined in different file.
KinBody.CheckSelfCollision has CollisionChecker parameter. CollisionChecker.CheckCollision has KinBody parameter. But they are defined in different file.

When there are mutual dependency, this method can be used:

py::class_ a declaration
py::class_ b declaration
a.def(b);
b.def(a);

However, if the files are separated, such dependency cannot be written.

cielavenir commented 1 year ago

The current state is this

mujin@desktop105:~/tmp $ python3 -m pybind11_stubgen --ignore-invalid=all openravepy
[2023-06-13 00:25:21,457] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,457] {__init__.py:149} WARNING - def ExtractAffineValues(self: openravepy._openravepy_.openravepy_int.ConfigurationSpecification, data: object, body: openravepy::PyKinBody, affinedofs: int, timederivative: int = 0) -> object: ...
[2023-06-13 00:25:21,457] {__init__.py:150} WARNING -                                                                                                                                ^-- Invalid syntax
[2023-06-13 00:25:21,458] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,458] {__init__.py:149} WARNING - def ExtractJointValues(self: openravepy._openravepy_.openravepy_int.ConfigurationSpecification, data: object, body: openravepy::PyKinBody, indices: object, timederivative: int = 0) -> object: ...
[2023-06-13 00:25:21,458] {__init__.py:150} WARNING -                                                                                                                               ^-- Invalid syntax
[2023-06-13 00:25:21,458] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,458] {__init__.py:149} WARNING - def ExtractTransform(self: openravepy._openravepy_.openravepy_int.ConfigurationSpecification, transform: object, data: object, body: openravepy::PyKinBody, timederivative: int = 0) -> object: ...
[2023-06-13 00:25:21,458] {__init__.py:150} WARNING -                                                                                                                                                ^-- Invalid syntax
[2023-06-13 00:25:21,459] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,459] {__init__.py:149} WARNING - def ExtractUsedIndices(self: openravepy._openravepy_.openravepy_int.ConfigurationSpecification, env: openravepy::PyKinBody) -> object: ...
[2023-06-13 00:25:21,459] {__init__.py:150} WARNING -                                                                                                                ^-- Invalid syntax
[2023-06-13 00:25:21,461] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,461] {__init__.py:149} WARNING - def InsertJointValues(self: openravepy._openravepy_.openravepy_int.ConfigurationSpecification, data: object, values: object, body: openravepy::PyKinBody, indices: object, timederivative: int = 0) -> bool: ...
[2023-06-13 00:25:21,461] {__init__.py:150} WARNING -                                                                                                                                              ^-- Invalid syntax
[2023-06-13 00:25:21,470] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,470] {__init__.py:149} WARNING - def Init(self: openravepy._openravepy_.openravepy_int.Controller, arg0: openravepy::PyRobotBase, arg1: str) -> bool: ...
[2023-06-13 00:25:21,470] {__init__.py:150} WARNING -                                                                                   ^-- Invalid syntax
[2023-06-13 00:25:21,470] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,470] {__init__.py:149} WARNING - def Init(self: openravepy._openravepy_.openravepy_int.Controller, robot: openravepy::PyRobotBase, dofindices: object, controltransform: int) -> bool: ...
[2023-06-13 00:25:21,470] {__init__.py:150} WARNING -                                                                                    ^-- Invalid syntax
[2023-06-13 00:25:21,471] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,471] {__init__.py:149} WARNING - def SetPath(self: openravepy._openravepy_.openravepy_int.Controller, arg0: openravepy::PyTrajectoryBase) -> bool: ...
[2023-06-13 00:25:21,471] {__init__.py:150} WARNING -                                                                                      ^-- Invalid syntax
[2023-06-13 00:25:21,534] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,534] {__init__.py:149} WARNING - def GetInterfaceType(self: openravepy._openravepy_.openravepy_int.Interface) -> OpenRAVE::InterfaceType: ...
[2023-06-13 00:25:21,534] {__init__.py:150} WARNING -                                                                                          ^-- Invalid syntax
[2023-06-13 00:25:21,536] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,536] {__init__.py:149} WARNING - def SetUserData(self: openravepy._openravepy_.openravepy_int.Interface, data: openravepy::PyUserData) -> None: ...
[2023-06-13 00:25:21,536] {__init__.py:150} WARNING -                                                                                         ^-- Invalid syntax
[2023-06-13 00:25:21,536] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,536] {__init__.py:149} WARNING - def SetUserData(self: openravepy._openravepy_.openravepy_int.Interface, key: str, data: openravepy::PyUserData) -> None: ...
[2023-06-13 00:25:21,536] {__init__.py:150} WARNING -                                                                                                   ^-- Invalid syntax
[2023-06-13 00:25:21,609] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,609] {__init__.py:149} WARNING - def CheckSelfCollision(self: openravepy._openravepy_.openravepy_int.KinBody, report: openravepy::PyCollisionReport = None, collisionchecker: openravepy::PyCollisionCheckerBase = None) -> bool: ...
[2023-06-13 00:25:21,609] {__init__.py:150} WARNING -                                                                                                ^-- Invalid syntax
[2023-06-13 00:25:21,633] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,633] {__init__.py:149} WARNING - def SetSelfCollisionChecker(self: openravepy._openravepy_.openravepy_int.KinBody, collisionchecker: openravepy::PyCollisionCheckerBase) -> None: ...
[2023-06-13 00:25:21,633] {__init__.py:150} WARNING -                                                                                                               ^-- Invalid syntax
[2023-06-13 00:25:21,865] {__init__.py:1158} INFO - Useful link: Avoiding C++ types in docstrings:
[2023-06-13 00:25:21,865] {__init__.py:1159} INFO -       https://pybind11.readthedocs.io/en/latest/advanced/misc.html#avoiding-cpp-types-in-docstrings

cielavenir commented 1 year ago

@felixvd @gywn we might need to redesign init_openravepy_kinbody() etc, please check..

cielavenir commented 1 year ago

[2023-06-13 00:25:21,633] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,633] {__init__.py:149} WARNING - def SetSelfCollisionChecker(self: openravepy._openravepy_.openravepy_int.KinBody, collisionchecker: openravepy::PyCollisionCheckerBase) -> None: ...
[2023-06-13 00:25:21,633] {__init__.py:150} WARNING -                                                                                                               ^-- Invalid syntax

This one can be removed by 8bdb91f97, but I'm not sure if such PyCollisionCheckerBaseBinderPtr is a good approach

[Edit] I kindof mean by redesign init_openravepy_kinbody proposing alternative approach(es) to PyCollisionCheckerBaseBinderPtr.

cielavenir commented 1 year ago

7a7df6f2ddc6770c5da0020a24dc9b7772213fb3 is much cleaner (see https://github.com/rdiankov/openrave/commits/fixStubgen_binderPtr)

cielavenir commented 1 year ago

[2023-06-13 00:25:21,609] {__init__.py:142} WARNING - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2023-06-13 00:25:21,609] {__init__.py:149} WARNING - def CheckSelfCollision(self: openravepy._openravepy_.openravepy_int.KinBody, report: openravepy::PyCollisionReport = None, collisionchecker: openravepy::PyCollisionCheckerBase = None) -> bool: ...
[2023-06-13 00:25:21,609] {__init__.py:150} WARNING -

this one got fixed too

cielavenir commented 1 year ago

further fix generated the very first typestub

openravepy_pybind11_stubgen_20230613.zip

rdiankov / openrave

WIP: Fix stubgen #1254

Background

Result