Embedded python: Pass an C++ object by reference

[singhanubhav2000]

I have a use-case where I have implemented an API in python and that needs to be invoked by C++.

struct ABC { ABC(int _i, int _j) { i = _i; j = _j; } int i; int j; };

struct listABC { listABC() {} std::string name; std::vector lst; };

PYBIND11_MAKE_OPAQUE(std::vector); PYBIND11_MODULE(cc11binds, m) { py::class(m, "ABC") .def(py::init<double, double>()) ; py::class_(m, "listABC") .defreadwrite("name", &listABC::name) .defreadwrite("lst", &listABC::lst) ; py::bind_vector<std::vector>(m, "ABCTest", py::module_local(false)); }

struct listABC { std::string name; std::vector lst; };

cumabc(&abcList) -> this takes reference to vector object. This works fine:

before cumabc, in C++ --- 2 [i:1 :: j:2] after cumabc, in C++ --- 2 [i:1 :: j:2, i:3 :: j:4] Where (3,4) is added by python interpreter.

cumlistABC(&listAbcWrapper) -> This takes reference to listABC object.

My code encounters both the type of pass by reference. Both listAbcWrapper and abcList are passed by class A to B. I was curious if we need to manage the reference count of listAbcWrapper and abcList (i.e. increase the reference count till their lifetime in A. A can later call a freelistAbc and freeCumAbc). My architecture is:	-----------------		------------------		--------------------		Main Driver A	-->	C++ Wrapper B	----->	Python Callee C

[vanossj]

is this different that #1508? Code doesn't compile, need a more complete example.

[singhanubhav2000]

The code is similar but this is a different discussion. All I wanted was how to manage reference count for the scenario I mentioned above.

[vanossj]

If you pass around pybind11::object objects, then the reference counting is handled for you.

If you use the python c api, like raw PyObject, or pybind11::handle, then you need to handle reference counting for those objects manually.

[singhanubhav2000]

I have a problem where the code is panicing when I am making python call from C++. Its on the line of the above discussion: the python snippet is import utils import TplatPythonInterface

class TplatPyModule: def init(self): pass

    def connect(self, tplatEndpointInfo):
        self.cls = utils.getClassByName(tplatEndpointInfo.plugin_path,
                                        tplatEndpointInfo.sub_plugin,
                                        tplatEndpointInfo.application)
        self.cls.connect(tplatEndpointInfo.endpoint_ip, tplatEndpointInfo.blob)

Here is binding code

include "embed.h"

include "stl_bind.h"

include "tplat_py_wrapper.h"

include "sma_interface.h"

namespace py = pybind11;

Here epInfo_(of type tplatEndpointInfo*) is defined in the base class smaTplatInterface,

class attribute((visibility("hidden"))) tplatPyWrapper : public smaTplatInterface { static pybind11::module module; pybind11::object pyModule; <-- Pointer to python class static pybind11::object import(const std::string& module, const std::string& path, pybind11::object& globals); public: static void initialize(); static void finalize(); tplatPyWrapper(tplatEndpointInfo* epInfo); ~tplatPyWrapper() {}

SmaStatus connect();

};

py::module tplatPyWrapper::module_;

PYBIND11_MAKE_OPAQUE(tplatEndpointInfo);

PYBIND11_EMBEDDEDMODULE(TplatPythonInterface, m) { m.doc() = "Thirdplatform framework to python binding"; py::class(m, "TplatEndpointInfo", py::call_guard()) .def(py::init<>(), py::return_value_policy::reference_internal) .defreadwrite("application", &tplatEndpointInfo::application, py::return_value_policy::reference_internal) .def_readwrite("endpoint_path", &tplatEndpointInfo::endpointpath, py::return_value_policy::reference_internal) .def_readwrite("endpoint_ip", &tplatEndpointInfo::endpointip, py::return_value_policy::reference_internal) .def_readwrite("sub_plugin", &tplatEndpointInfo::subplugin, py::return_value_policy::reference_internal) .def_readwrite("plugin_path", &tplatEndpointInfo::pluginpath, py::return_value_policy::reference_internal) .defreadwrite("blob", &tplatEndpointInfo::blob, py::return_value_policy::reference_internal)

The code is panicing at the time when we are trying to call the connect function as defined under.

static void tplatPyWrapper::initialize() { py::initializeinterpreter(); py::object main = py::module::import("main"); py::object globals = main.attr("dict"); module = py::module::import("tplatPyModule"); }

tplatPyWrapper::tplatPyWrapper() { auto tplatPyModuleClass = module.attr("TplatPyModule"); pyModule = tplatPyModuleClass(); }

SmaStatus tplatPyWrapper::connect() { auto connect = pyModule.attr("connect"); connect(epInfo); return SMAOK; } initialize(); tplatPyWrapper* tp = new tplatPyWrapper(epInfo); tp->connect();

(gdb) bt

0 0x00007ffff57845f7 in raise () from /lib64/libc.so.6

1 0x00007ffff5785ce8 in abort () from /lib64/libc.so.6

2 0x00007ffff60889d5 in __gnu_cxx::__verbose_terminate_handler() () from /lib64/libstdc++.so.6

3 0x00007ffff6086946 in ?? () from /lib64/libstdc++.so.6

4 0x00007ffff6086973 in std::terminate() () from /lib64/libstdc++.so.6

5 0x00007ffff6086b93 in _cxa_throw () from /lib64/libstdc++.so.6

6 0x00000000008f32a3 in pybind11::detail::simple_collector<(pybind11::return_value_policy)1>::call (this=0x7fffef9b4950, ptr=0xdbe690) at ../../../common/pybind11/cast.h:1943

7 0x00000000008f00ee in pybind11::detail::object_api::operator()<(pybind11::return_value_policy)1, tplatEndpointInfo*&> (this=0x7fffef9b4990) at ../../../common/pybind11/cast.h:2096

8 0x00000000008d97c2 in tplatPyWrapper::connect (this=0x7fffcc01b6c0) at tplat_py_wrapper.cc:130

[singhanubhav2000]

Hi Guys, Any idea how to fix the issue? Any help would be greatly appreciated.