Installation problem in Ubuntu 14.04

[qmaxquique]

As I'm unable to use Docker, I'm trying to install MOE in a clean Ubuntu14.04 vm. I was able to clone the repo and install the python dependencies as needed but when running the setup.py I get:

root@ubuntu:~/MOE# python setup.py install
.
. <output shortened>
.
running install_egg_info
removing '/usr/local/lib/python2.7/dist-packages/MOE-0.2.1.egg-info' (and everything under it)
Copying MOE.egg-info to /usr/local/lib/python2.7/dist-packages/MOE-0.2.1.egg-info
running install_scripts
Could not find env var MOE_CMAKE_PATH for cmake, using /usr/bin/cmake from $PATH
MOE_CMAKE_OPTS not set. Passing no extra args to cmake.
PYTHON_INCLUDE_DIR (Expected full path to where Python.h is found): /usr/include/python2.7
PYTHON_LIBRARY (Expected path to Python shared object; e.g., libpython2.7.so or .dylib): /usr/lib/libpython2.7.so.1.0
-- The C compiler identification is GNU 4.8.2
-- The CXX compiler identification is GNU 4.8.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
CMake Error at CMakeLists.txt:62 (message):
  MOE_PYTHON_LIBRARY is not a valid file: /usr/lib/libpython2.7.so.1.0

-- Configuring incomplete, errors occurred!
See also "/root/MOE/moe/build/CMakeFiles/CMakeOutput.log".
make: *** No targets specified and no makefile found.  Stop.
error: [Errno 2] No such file or directory: '/root/MOE/moe/build/GPP.so'

In case you want to take a look, I've created a machine at Terminal.com showing the issue and I've granted access to any *@yelp.com e-mail. If you don't have an account at Terminal.com you can create one in less than a minute and you will also get some free credit to play with.

This is the VM where I was trying to install MOE: https://terminal223.terminal.com/

Any help is highly appreciated. Thank you!

[sc932]

Thanks for pointing this out @qmaxquique, we were able to repro this and have a fix being prepared now. I'll update this thread when when it is done.

In the very short term you can get around this by deleting lines 190 and 191 from setup.py

            # '-DMOE_PYTHON_INCLUDE_DIR=' + moe_python_include_dir,
            # '-DMOE_PYTHON_LIBRARY=' + moe_python_library,

[qmaxquique]

Thank you!! I will make a new snapshot with MOE ready to be used at Terminal.com. It could be useful for you guys as it will provide a full configured VM with MOE in just a few seconds. If you need a hand just let me know! Cheers, Enrique

[qmaxquique]

Hi, Did you guys solved this problem already? We're working on an installation script for MOE and we would like to keep it as clean as possible.

Thanks, Enrique

[dgreis]

Hey @sc932, any updates on this? I tried your workaround, but still see this:

ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status make[2]: * [GPP.so] Error 1 make[1]: * [CMakeFiles/GPP.dir/all] Error 2 make: *\ [all] Error 2 error: [Errno 2] No such file or directory: '/Users/my.user.name/Documents/git_repos/MOE/moe/build/GPP.so'

I'm on OSX and am using the latest version of MOE. Would appreciate your help.

[suntzu86]

Sorry I've been gone for so long! See: #448 for details.

@qmaxquique I'm sorry we totally failed to follow up on this :( It's been so long I honestly have no recollection of what fix we thought we had found. I have a very vague memory that the issue was caused by cmake searching in the wrong spot for that library. Like ubuntu added another folder on that path so it was /usr/lib/some_folder/libpython2.7.so.1.0 instead of /usr/lib/libpython2.7.so.1.0, and cmake wasn't finding it.

If this is still of interest to you, can you post your gcc & boost versions, as well as the result of locate libpython2.7? (Might have to sudo updatedb first)

@dgreis not sure that's a related issue. What OSX are you on? I googled the first line of your pasted output and found this: https://github.com/bitcoin/bitcoin/issues/3228

[nbalakumaran]

Sorry for reviving the dormant thread. Just jotting down my fix

Commenting line 190 and 191 in setup.py seems to work. But there is a small issue of cmake detecting the wrong version of python libs. This seems like a prevelant issue with cmake and python.

Instead of removing the cmake flags i hard coded the value i.e. -DMOE_PYTHON_LIBRARY=/path/to/libpython2.7.so.1.0

Also, other issue of MOE_PYTHON_LIBRARY is not a valid file: /usr/lib/libpython2.7.so.1.0 can be fixed by either symlinking or setting the flag to the location of libpython2.7.so.1. Most probably it would be /usr/lib/x86_64-linux-gnu/libpython2.7.so.1

[Riaz123]

Hi @nbalakumaran , where do I set flag to the location of libpython2.7.so.1 ? is it in line 191 of setup.py? likw below line? '-DMOE_PYTHON_LIBRARY=' '/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0',

Even this does not work for me, i still get "No such file or directory : ./moe/build/GPP.so "

[Riaz123]

Would appreciate if anyone can share MOE setup.py for Ubuntu 14.0.

[wujian16]

We have used (and extended) Yelp MOE, https://github.com/wujian16/Cornell-MOE. There we have a detailed installment instruction on AWS EC2 with Ubuntu OS based on our experiences, specifically see step 3. We have not modified setup.py itself in anyway.

[miumm]

hi @wujian16 ,I failed to install sigopt,and then I successfully installed your Cornell-MOE，and I wonder are there some differences between sigopt with your MOE? thanks!

[wujian16]

Thanks for your interest. For the algorithm implemented in Yelp MOE, i.e. GP+EI, Cornell-MOE just copied from MOE. The only difference is Cornell-MOE treats the GP hyperparameters in a Bayesian way, which should not hurt the performance if not improve.

For Sigopt, as far as I know, they built upon MOE, but over the last 3 to 4 years, they also invented some in-house proprietary techniques which make the algorithm more robust and better in some cases. So I think that Yelp MOE cannot exactly match the performance of SigOPT (maybe better or worse), so does Cornell-MOE.