Issue with functions that have function pointers as parameters

[renautomation]

In my code, I have stuff like this:

typedef retCode_t (*netReqCallback_t)(const netReq_t event,
                                                             const std::string req,
                                                             const std::string data,
                                                             const int32_t n);

...

class MyClassGateway {
public:
...
    retCode_t regNetReqCallback(const netReqCallback_t callback);

Now, when I use litgen to expose regNetReqCallback to Python, the build is successful but in Python I get the following error:

>>> ret = regw.regNetReqCallback(network_callback)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: regNetReqCallback(): incompatible function arguments. The following argument types are supported:
    1. (self: MyClass._MyClass.MyClassGateway, callback: retCode_t (netReq_t, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)) -> MyClass._MyClass.retCode_t

Invoked with: <MyClass._MyClass.MyClassGateway object at 0x7f30dc0091f0>, <function network_callback at 0x7f30dbfed6c0>

Did you forget to `#include <pybind11/stl.h>`? Or <pybind11/complex.h>,
<pybind11/functional.h>, <pybind11/chrono.h>, etc. Some automatic
conversions are optional and require extra headers to be included
when compiling your pybind11 module.
>>>

Now, when in the past I worked with manual pybind11 module creation, I fixed this issue by using the approach suggested here: https://stackoverflow.com/questions/74480093/pybind11-pass-c-style-function-pointer-as-a-parameter

But how can I use the same approach by using litgen?

[pthom]

hello, I am the author of this SO question.

Concerning your issue, you will have to manually write a wrapper that uses a std::function, and use bindings for this wrapper (you could exclude also the original one via one of the exclude options)

[renautomation]

ahahah I didn't note that you were the author of that question... the related solution helped me so much!

Regarding your litgen solution, is the following procedure what you are suggesting?

Procedure:

• to exclude the original function MyClassGateway::regNetReqCallback with exclude options
• create a wrapper for MyClassGateway::regNetReqCallback in the pybind11 C++ module, using std::function instead of the function pointer type netReqCallback_t, and to put it before the autogenerated code section
• use litgen to generate the bindings for the wrapper and not for the original function.

If so, how can I tell litgen to generate the code for the wrapper function specified into the pybind11 module, instead of the original function in the header file processed by litgen.write_generated_code_for_files function?

[pthom]

Do not add your wrapper in the cpp pybind wrapper module, add it in an additional header that you tell litgen to also parse

[renautomation]

ok, coming back to this topic, I created a wrapper in a separated file Wrapper.h, that wraps the class member function regNetReqCallback that has the function pointer as parameter:

retCode_t regNetReqCallbackWrapper(MyClassGateway &self, std::function<std::remove_pointer_t<netReqCallback_t>> stdCallback) {
    static std::function<std::remove_pointer_t<netReqCallback_t>> callback = std::move(stdCallback);
    return self.regNetReqCallback(
        [](const netReq_t event, 
           const std::string req, 
           const std::string data, 
           const int32_t n) -> retCode_t {
                return callback(event, req, data, n);
        });
}

After adding a couple of litgen options, it is now able to build and run on Python side, however I have to call the function like this:

ret = regNetReqCallbackWrapper(my_class_gateway_object, network_callback)

I'm wondering if there is a way for litgen, in the pybind module, to include the wrapper directly into the MyClassGateway::regNetReqCallback binding, so that in Python I could do:

ret = my_class_gateway_object.regNetReqCallbackWrapper(network_callback)

When in the past I worked with manual implementation of pybind11 module, I did it in this way:

py::class_<MyClassGateway>(m, "MyClassGateway")
    .def(py::init([]()  {
        < ... some code ... >
    }),
    .def("regNetReqCallback", [](MyClassGateway &self, std::function<std::remove_pointer_t<netReqCallback_t>> stdCallback) {
        static std::function<std::remove_pointer_t<netReqCallback_t>> callback = std::move(stdCallback);
        return self.regNetReqCallback(
            [](const netReq_t event, const std::string req, const std::string data, const int32_t n) -> retCode_t {
                return callback(event, req, data, n);
            });
    }, py::arg("callback"))

but now I don't know how to instruct litgen to do that for me (if possible). I ask also because, for another class, I have a similar situation, i.e. function pointers as parameters, directly in the class constructor, so I really would like to instruct litgen to do that.

[pthom]

litgen will never be able to understand

std::function<std::remove_pointer_t<netReqCallback_t>>

This is advanced C++ dark black magic

You should use std::function with concrete params.