caffe2 installs Python library to wrong location relative to CMAKE_INSTALL_PREFIX

[ezyang]

I set CMAKE_INSTALL_PREFIX=$PREFIX and caffe2 dumped the Python modules in $PREFIX and not in $PREFIX/lib/python2.7 (or perhaps site-packages) where they should be placed. Subsequently, I had to add $PREFIX to my $PYTHONPATH to get caffe2 to find it.

This was tested on ae9af51496a4485437cf704bc7566eebe5e7f4d6

I used this conda build.sh and meta.yaml to do the build:

#!/bin/bash
set -e
if [ -z "$CONDA_CUDA_HOME" ]; then
  CUDA_ARGS=""
else
  CUDA_ARGS="-DCUDA_TOOLKIT_ROOT_DIR=$(printf %q "$CONDA_CUDA_HOME")"
fi
if [ -z "$PREFIX" ]; then
  PREFIX="$CONDA_PREFIX"
fi
rm -rf build || true
PYTHON_ARGS="$(python ./scripts/get_python_cmake_flags.py)"
mkdir -p build
cd build
cmake3 -DCMAKE_INSTALL_PREFIX="$PREFIX" $CUDA_ARGS $PYTHON_ARGS ..
make -j20
make install/fast

and

{% set version = "0.8.dev" %}

package:
  name: caffe2
  version: {{ version }}

source:
  path: ..

build:
  number: 1
  skip: True  # [win]

requirements:
  build:
    - future
    - glog
    - numpy
    - six
    - eigen
    - python <3
  run:
#   - future
#   - glog
#   - numpy
#   - six
#   - eigen
#   - python ==2.*

about:
  home: https://caffe2.ai/
  license: BSD
  summary: deep learning library

extra:
  recipe-maintainers:
    - ezyang

[lukeyeager]

It seems like CMake should be able to automatically infer the correct installation prefix based on the base installation prefix, but as far as I can tell it can't.

One person solved this issue by using setuptools:

I realized that I could just call the standard Python installer from CMake. This led to a more reliable install process and also a vastly cleaner CMakeLists.txt file. My line for installing the Python code is simply: install(CODE "execute_process(COMMAND python setup.py install -f --prefix=${CMAKE_INSTALL_PREFIX} WORKING_DIRECTORY ../source/libmoleculizer-1.1.2/python-src/language_parser)") http://public.kitware.com/pipermail/cmake/2013-March/054141.html

I wish CMake had something like GNUInstallDirs, but for Python: https://github.com/BVLC/caffe/pull/4237

/cc @CDLuminate

[ezyang]

Yes, I agree that using setuptools is a reasonable way to solve the problem. Caffe2 doesn't have a setup.py yet but it ought not be hard to write.