No module named caffe2_pybind11_state (Mac)

[knipping]

Everything was running fine when I installed Caffe2 from source last week on my Mac (Sierra).

I updated the installation (pull, make clean, cmake, make install) today. Now when I run my networks or just call python -c 'from caffe2.python import core' I receive the fatal error CRITICAL:root:Cannot load caffe2.python. Error: No module named caffe2_pybind11_state. I located caffe2_pybind11_state.so in /usr/local/caffe2/python/ but failed to make it available to python so far.

The cause for this is likely the make install message I got at the very end:

error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: for: 
caffe2_repo_path/build/caffe2/python/caffe2_pybind11_state.so (for architecture x86_64)
option "-add_rpath /usr/local/lib" would duplicate path, 
file already has LC_RPATH for: /usr/local/lib`

[knipping]

After fiddling a bit around with the -d option of make and locating the install_name_tool call I managed to fix this. The call is in caffe2_repo_path/build/caffe2/CMakeFiles/caffe2_pybind11_state.dir/build.make, final rule for caffe2/python/caffe2_pybind11_state.so:.

According to the error message the problem is to try adding a path to caffe2_pybind11_state.sowhich is already present there. This error causes a deletion of the new caffe2_pybind11_state.so as well as skipping any remaining installs.

I just commented cd caffe2_repo_path/build/caffe2 && install_name_tool -add_rpath /usr/local/lib caffe2_repo_path/build/caffe2/python/caffe2_pybind11_state.so in the above mentioned build.make. Now it seems to do a proper install and my Caffe2 programs run.

[knipping]

Sorry, closed this by accident. While I got it working now for me I guess this still need fixing in the make process.

[aaronmarkham]

So this problem arises if you have already installed Caffe2 and have the python binding and then the call in CMakeLists.txt is breaking things the second time around? What's the proposed change to make it work the first time and not break the second + time? Add -d?

cc @bwasti

[knipping]

Sorry if my post was misleading, I just used the -d for the call to make to locate the problem.

I think the problem occurs if caffe2_pybind11_state.so already contains the path /usr/local/lib. Why it contains the path already is unclear to me. However, it happens to me also for a cleaned build directory and even for the very first build call on a fresh setup, no Caffe2 installed -- I tried again on another machine. The only difference from a standard Mac installation I am aware of is that I used port instead of homebrew. (We use Mac ports a lot in our company and I throwing homebrew into the mix may easily break some critical systems).

As far as I know there is no option for install_name_tool to suppress an error if the path already exists an I am not ware of a simple check for a path already being contained for skipping the action in that case. So, unless there is an better idea, I would suggest to make the error here non-fatal for the build process and just report a warning.

[aktiger]

I copy caffe2_pybind11_state.so and caffe2_pybind11_state_gpu.so to the caffe2 path: cp /usr/local/caffe2/python/caffe2_pybind11_state.so /home/cdsl/zhangchao/caffe2/caffe2/python/ cp /usr/local/caffe2/python/caffe2_pybind11_state_gpu.so /home/cdsl/zhangchao/caffe2/caffe2/python/

and change my .bashrc as follows : export PYTHONPATH="/home/cdsl/zhangchao/caffe2":$PYTHONPATH export PYTHONPATH="/usr/local/caffe2/python/":$PYTHONPATH export LD_LIBRARY_PATH="/usr/local/caffe2/python/":"/home/cdsl/zhangchao/caffe2/build/lib":$LD_LIBRARY_PATH

then everything works ok now!

[huhai463127310]

@aktiger It works. thx

[mazhen18]

[solved] when I install in python2.7 then install same pytorch in python3.6, " No module named caffe2_pybind11_state" happend. So, I re-download pytorch from github by this command "git clone --recursive https://github.com/pytorch/pytorch", then install from source again. Everything is ok.

By the way, the command to install pytorch on mac I used was command "MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python3.6 setup.py install".

The caffe2_pybind11_state module in python2.7 named "caffe2_pybind11_state.so", while in python3.6 may named like "caffe2_pybind11_state.cpython-36m-darwin.so"