“No module named caffe” but .bashrc references the location, Makefile.config seems fine

[mxchinegod]

So I got this error when running a python script that wraps around caffe. I was like "oh right, go fix .bashrc so it knows what you're trying to do". A la

export PYTHONPATH=/home/ubuntu/caffe/python

(also later down here it gives a similar error for CUDA, and the paths for that are also correct in .bashrc making me suspect there is a larger issue here)

it still doesn't work. I tried the test scripts that come with caffe so it's fine and works properly but just not through python importing. Nothing went wrong during compiling caffe and my Makefile.config directs to the library correctly by default unless I'm missing something. I can paste that as well (this is ubuntu 14.04).

# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
                /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
                # $(ANACONDA_HOME)/include/python2.7 \
                # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

Lastly...

>>> import sys 
>>> sys.path.append('/home/ubuntu/caffe/python') 
>>> import caffe 

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/ubuntu/caffe/python/caffe/__init__.py", line 1, in <module> from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver File "/home/ubuntu/caffe/python/caffe/pycaffe.py", line 13, in <module> from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \ ImportError: libcudart.so.7.5: cannot open shared object file: No such file or directory –

I checked, everything is in the right place, last thing I checked was:

cd /home/ubuntu/caffe/

and then

sudo pip install -r ./python/requirements.txt

And it installed some updates but nothing changed. I've restarted terminals, etc. Restarted, sourced bashrc, "make pycaffe" and nothing needs to be done it says.

What's up? Am I crazy? Not using Anaconda or anything, just py2.7 default + satisfied requirements from the .txt

[lukeyeager]

Check your ImportError message more carefully:

ImportError: libcudart.so.7.5: cannot open shared object file: No such file or directory

You probably need to set up your environment to find CUDA: https://github.com/NVIDIA/DIGITS/blob/digits-4.0/docs/InstallCuda.md#environment

EDIT

Oh, I see you already noticed that. Sorry.

[mxchinegod]

I'm excited to read and hopefully fix the error emergent from using sys to append a path in the python environment.

That being said, is there a reason/way to fix the issue preceding that whereby it can't find caffe? I notice caffe does not install itself into site-packages... (not sure if I doublechecked this on my own system but I've read this and it would explain all of the references I must make)

[mxchinegod]

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
export PATH=$PATH:/usr/local/cuda-7.5/bin
export LD_LIBRARY_PATH=:/usr/local/cuda-7.5/lib64
export PYTHONPATH=/home/ubuntu/caffe/python:$PYTHONPATH

my ~/.bashrc fyi

[lukeyeager]

I notice caffe does not install itself into site-packages...

See discussion here: https://github.com/BVLC/caffe/issues/3925

[mxchinegod]

Well I'm not really inquiring as to the reasons or how to make it more of a traditional package to be frank... I just want to know why it can't find anything when my environment variables seem wholesome. I'm fine with it being quirky and different if it works :P

EDIT: so the Nvidia CUDA environment guide helped and a live python environment using sys.path.append now allows for caffe to be imported as well but if I'm using a wrapper, I have to do this every time?

[lukeyeager]

Oh I see. Sorry, I didn't read your OP very carefully :astonished:

Dunno. Your setup looks pretty standard to me.

[mxchinegod]

Dunno. Your setup looks pretty standard to me.

Well that's good news at least!

EDIT: I ended up getting it to work by modifying some code in a script and caffe to fit my situation. Thanks for the help. It's all working now.

[seanbell]

Glad you got it working.

[sayadyaghoobi]

i got the same error! anyone has solved this please give me the point. i compiled correctly but still it give me no caffe module found. i checked paths for cuda and caffe, both of them were right. please help me with this issue:

export PATH=/usr/local/cuda-8.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH export CAFFE_ROOT=/Caffe export PYTHONPATH=/Caffe/python:$PYTHONPATH

[selinachenxi]

@sayadyaghoobi I got the same error. Did you solve it?

[sayadyaghoobi]

@selinachenxi i just added a caffe's python path to my python environment :

import sys

sys.path.append('/Caffe/python')

import caffe

try this, if you face the errors (whatever it works or not), just email me, feel free

[selinachenxi]

Hi, I tried many times, just doesn't work. I even copied the python folder under the sitepackages in anaconda, still doesn't work. I am thinking might be the version problem. I am using the caffe provided by the openpose code. It only generates _caffe.so, but not pycaffe.pyc etc. I have no problem to install in my own computer, just not on the server without root.

On 15 June 2017 at 07:06, sayadyaghoobi notifications@github.com wrote:

hi, yes! i just added the caffe's python directory to my python environment : import sys

sys.path.append('/Caffe/python')

import caffe

then it solved! try this if you got an errors just email me (whatever it works or not), feel free.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BVLC/caffe/issues/4389#issuecomment-308637787, or mute the thread https://github.com/notifications/unsubscribe-auth/ALM45AQKC6dUdyr7h3V_TfH8f2aeHJuxks5sEMnigaJpZM4JBYiG .

[sayadyaghoobi]

are you sure that pycaffe compiled ? make pycaffe

Add the module directory to your $PYTHONPATH by export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH or the like for import caffe.

[selinachenxi]

I am not sure in fact. Because I am using 3rd party provided caffe, it might have problem about their make for pycaffe. I shall try on the official version.

On 15 June 2017 at 12:34, sayadyaghoobi notifications@github.com wrote:

are you sure that pycaffe compiled ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BVLC/caffe/issues/4389#issuecomment-308705242, or mute the thread https://github.com/notifications/unsubscribe-auth/ALM45OUnfZvygwyFgizE-pzuvD2c3LBKks5sERbIgaJpZM4JBYiG .