connected pybind11 modules

[GliderGeek]

I think my problem is not necessarily a bug, but asking before on SO, gitter and an old related issue, didn't lead to solutions which solved the issue. Futhermore i think this use case might come up more frequently and can thus be of use for others if documented.

Issue description

Am trying to create and bind the following two modules with pybind11:

• point: can be called directly
• line: can be called directly and also uses point module. Trying to use use the line module leads to an undefined symbol error: Symbol not found: __ZN5pointC1Eddd

Reproducible example code

The original setup is as follows:

point.h:

#ifndef UNTITLED1_POINT_H
#define UNTITLED1_POINT_H

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

class Point {

private:
    double m_x;
    double m_y;
    double m_z;

public:
    Point()= default;
    Point(double x, double y, double z);
};

PYBIND11_MODULE(point, m) {

    py::class_<Point>(m, "Point")
            .def(py::init<double, double, double>());

}

#endif //UNTITLED1_POINT_H

point.cpp:

#include "point.h"
Point::Point (double x, double y, double z){
    m_x = x;
    m_y = y;
    m_z = z;
}

line.h:

#ifndef UNTITLED1_LINE_H
#define UNTITLED1_LINE_H

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

#include "point.h"

class Line {
private:
    Point m_p1;
    Point m_p2;
public:
    Line(Point p1, Point p2);
};

PYBIND11_MODULE(line, m) {

    py::class_<Line>(m, "line")
            .def(py::init<Point, Point>());

}

#endif //UNTITLED1_LINE_H

line.cpp:

#include "line.h"

Line::Line(Point p1, Point p2) {
    m_p1 = p1;
    m_p2 = p2;
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.0)
project(untitled1)

set(CMAKE_CXX_STANDARD 11)

add_subdirectory(pybind11)

pybind11_add_module(point point.cpp)
pybind11_add_module(line line.cpp)

Now the following python code is run:

from point import point
from line import line

p1 = point(1, 2, 3)
p2 = point(3, 4, 5)

l = line(p1, p2)

leading to a undefined symbol error: Symbol not found: __ZN5pointC1Eddd

I have also tried the following lines in the cmake file: pybind11_add_module(point SHARED point.cpp) pybind11_add_module(line line.cpp) target_link_libraries(line PRIVATE point)

Also tried adding different linking operations: py::module::import("point"); py::object point = (py::object) py::module::import("point"); py::module point = py::module::import("point");

[daniel-ziegler]

I'd also like to be able to do this, although I'm currently building with setuptools instead of CMake.

[TallJimbo]

Are you also building regular C++ shared libraries for point.cpp and line.cpp, and linking them as you would if you didn't have any Python bindings? While there may be a way to use pybind11 modules to pass regular C++ implementation symbols between translation units, the normal way of building pybind11 modules (i.e. with hidden symbols) will not do this. It might work without additional libraries if you explicitly declare any symbols that need to be shared as having public visibility using attributes.

[wgledbetter]

I was having this problem earlier this week and stole a fix from a friend. Instead of creating separate "point" and "line" modules, you can compile them into one module. I know this might not make sense for all applications, but here's what it would look like in this case:

In both "point.h" and "line.h", change the module creation code into a function that creates a class when passed a module.

point.h

#ifndef UNTITLED1_POINT_H
#define UNTITLED1_POINT_H

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

class Point {

private:
    double m_x;
    double m_y;
    double m_z;

public:
    Point()= default;
    Point(double x, double y, double z);
};

// pybind11 binding function
void PB_point(py::module& m){
    py::class_<Point>(m, "Point")
        .def(py::init<double, double, double>());
}

#endif //UNTITLED1_POINT_H

line.h

#ifndef UNTITLED1_LINE_H
#define UNTITLED1_LINE_H

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

#include "point.h"

class Line {
private:
    Point m_p1;
    Point m_p2;
public:
    Line(Point p1, Point p2);
};

// pybind11 binding function
void PB_line(py::module& m){
    py::class_<Line>(m, "Line")
        .def(py::init<Point, Point>());
}

#endif //UNTITLED1_LINE_H

Then compile "point.cpp" and "line.cpp" to "point.o" and "line.o." I use g++ and pass all the arguments from https://pybind11.readthedocs.io/en/stable/basics.html

-O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes`

Then we need to create the module that contains "point" and "line." Call it "allMyCode."

allMyCode.cpp

#include "point.h"
#include "line.h"

#include <pybind11/pybind11.h>
namespace py = pybind11;

PYBIND11_MODULE(allMyCode, m){
    PB_point(m);
    PB_line(m);
}

Then, CMakeLists.txt is:

cmake_minimum_required(VERSION 3.0)
project(pb11)

set(CMAKE_POSITION_INDEPENDENT_CODE ON)
add_subdirectory(pybind11)

pybind11_add_module(allMyCode MODULE ./allMyCode.cpp)
target_link_libraries(allMyCode PRIVATE ./point.o)
target_link_libraries(allMyCode PRIVATE ./line.o)

Lastly, calling from python looks like

import allMyCode

pt1 = allMyCode.point(1,2,3)
pt2 = allMyCode.point(4,6,8)

ln1 = allMyCode.line(pt1,pt2)

Hope this helps!

[GiulioRomualdi]

Hi all, today I faced with exact same problem. So I played a bit with pybind11 Following what @GliderGeek wrote in the comment https://github.com/pybind/pybind11/issues/1391#issue-322103749 I created two libraries one called point and the other one line. Then I tried to wrap them independently.

In details in https://github.com/GiulioRomualdi/pybind11-multiple-modules you can find a complete example. The src folder contains the libraries point and line. These two libraries do not depend on pybind11 and are pure c++ libraries. The bindings folder contains the python bindings. In details, I create two separate modules point and line. Thanks to py::module::import("point");, I was able to build the two modules separately. Please check here.

Hope this will help you :smiley:

[adrn]

Running into exactly this issue, so I really appreciate all of the thoughts in this thread!

@GiulioRomualdi This is exactly how I have my package laid out, but trying to build with pybind11.setup_helpers.Pybind11Extension's instead. It builds fine, but I'm finding that I still get the undefined symbol error if I import (in Python) my equivalent "line" module first. I don't get an import error if I do

import point
import line

Any ideas here about this?