Import Error on M1 Mac when using Docker Image

[nimski]

First, thank you for all your work and support on binder.

I'm trying to run binder on a C++ project on an M1 Mac. To make building easier, I'm using the provided Dockerfile (based on Ubuntu 20.04) to quickly build binder. I then run it on a mounted host volume to generate the bindings from C++ files on the host. This all works well. However, when trying to import the generated module I get the following error:

dlopen(xxx.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__Z11bind_cpp_ttRNSt3__18functionIFRN8pybind117module_ERKNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEE'

This seems like an STL error. I'm wondering if this is because the binder built in the Docker image was built in an Ubuntu 20.04 environment (I'm assuming using libstdc++), and now I'm building its resulting bindings on an M1 Mac with apple clang (which I think uses libc++).

I've tried building the bindings on Mac with libstdc++ but it creates all sorts of errors and I believe is unsupported by Apple clang.

Is what I'm doing an unsupported use case? Building binder natively on Mac is not easy (especially as part of a CI build environment) so I'd love to be able to use the Docker image to do it instead.

Thanks!

[lyskov]

@nimski how exactly are you trying to build Binder on M1 Mac? Particularly have you tried using https://github.com/RosettaCommons/binder/blob/master/build.py script? (this is what i do when i run Binder on M1). One known issue with build on M1 Mac's is to use LLVM-13 or older (build.py script does that). - Could you please try it and see if it works for you?

re error above: in general recommended approach is to generate bindings and build them on the target platforms to avoid std lib incompatibility issues.

Let me know how it goes,

[nimski]

Thanks @lyskov! I tried building with build.py and this is what I'm getting. It looks like a configuration failure. Is build.py supported on M1 mac with Apple clang?

build_output.txt

[lyskov]

Thank you for detailed log @nimski - i will try to replicate this locally. To answer your question: - yes, build on M1 Mac's with Apple Clang is supported. In meanwhile: could you please double check that you are getting the same error on clean binder clone? Thanks,

[lyskov]

ok, my local build on clean Binder clone on M1 seems to work. @nimski could you please try building on clean clone and let me know if you are still seeing the error above? Thanks,

[nimski]

Thank you for the quick responses @lyskov! Building from source (on a clean clone) indeed solved the problem. Do you have a BuyMeACoffee account by any chance? I certainly owe you one!

[nimski]

It looks like this was not the end of the saga. Although building using build.py works well, I'm getting errors about missing headers:

fatal error: 'functional' file not found
[cmake] #include <functional>
[cmake]          ^~~~~~~~~~~~
[cmake] 1 error generated.

On the docs I see that the remedy for this is to point binder to the appropriate clang includes, but having tried a few different options I'm not sure what the correct include directory is. For example, if I try I/Library/Developer/CommandLineTools/usr/include/c++/v1 then I get a different error message:

fatal error: 'stdlib.h' file not found
[cmake] #include_next <stdlib.h>

Any pointers on how to fix this would be greatly appreciated!

[lyskov]

I routinely compile on Mac so it is certainly possible but might require a few tries to make include work. I would recommend to try adding this to your binder command line (at the end): binder <YOUR PROJECT ARGS> -- -x c++ -std=c++11 -isysroot `xcrun --show-sdk-path` and see if it helps.

[nimski]

Thanks so much! That fixed the problem.

[lyskov]

Great, glad to hear that!