search

DerThorsten / nifty

A nifty library for graph based image segmentation.
MIT License
41 stars 21 forks source link

Installation/build issue - Any doc for installation steps? #132

Open arinik9 opened 4 years ago

arinik9 commented 4 years ago

First, I would like to thank you for this github repo! Thanks to python API its use should be easier. This is why I would like to install the program through anaconda (as you recommend). If possible, I would like to ask you to provide some installation steps for:

In the following, I explain what I tried since yesterday: According to the readme file, the program is supposed to run with one simple line command: GUROBI_ROOT_DIR=/path/to/gurobi conda install -c cpape nifty-with-gurobi. However, it did not work out, since it does not found the package nifty-with-gurobi in the current channel of anaconda. My second attempt was to build nifty by using this command conda build conda-recipe. However, I had many issues during build process. The last one was this one:

In file included from /home/nejat/anaconda3/conda-bld/nifty_1582733307969/work/include/nifty/graph/rag/detail_rag/compute_grid_rag.hxx:22:0, from /home/nejat/anaconda3/conda-bld/nifty_1582733307969/work/include/nifty/graph/rag/grid_rag.hxx:19, from /home/nejat/anaconda3/conda-bld/nifty_1582733307969/work/include/nifty/graph/rag/grid_rag_features.hxx:3, from /home/nejat/anaconda3/conda-bld/nifty_1582733307969/work/src/python/lib/graph/rag/graph_accumulator.cxx:6: /home/nejat/anaconda3/conda-bld/nifty_1582733307969/work/include/nifty/z5/z5.hxx:4:10: fatal error: z5/dataset_factory.hxx: No such file or directory

include "z5/dataset_factory.hxx"

^~~~~~~~ compilation terminated. compilation terminated. src/python/lib/graph/rag/CMakeFiles/_rag.dir/build.make:110: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/grid_rag_stacked.cxx.o' failed make[2]: [src/python/lib/graph/rag/CMakeFiles/_rag.dir/grid_rag_stacked.cxx.o] Error 1 src/python/lib/graph/rag/CMakeFiles/_rag.dir/build.make:134: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/graph_accumulator.cxx.o' failed make[2]: [src/python/lib/graph/rag/CMakeFiles/_rag.dir/graph_accumulator.cxx.o] Error 1 CMakeFiles/Makefile2:1372: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/all' failed make[1]: [src/python/lib/graph/rag/CMakeFiles/_rag.dir/all] Error 2 [ 10%] Linking CXX shared module _nifty Copying target opt to temporary module directory [ 10%] Built target _opt Copying target nifty to temporary module directory [ 10%] Built target _nifty Makefile:140: recipe for target 'all' failed make: [all] Error 2 Traceback (most recent call last): File "/home/nejat/anaconda3/bin/conda-build", line 11, in sys.exit(main()) File "/home/nejat/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 438, in main execute(sys.argv[1:])

Thanks in advance! Nejat

DerThorsten commented 4 years ago

@constantinpape can maybe help, also his fork (https://github.com/constantinpape/nifty) is way more up to date then mine, but on a first glence it looks like z5 is missing which can be found here https://github.com/constantinpape/z5

arinik9 commented 4 years ago

Apparently, it can not find where numpy and z5 are installed (or, at least their header files). However, the header files of numpy are installed in: /home/nejat/anaconda3/envs/py37/lib/python3.7/site-packages/numpy/core/include/numpy, and those of z5 are installed in /home/nejat/anaconda3/envs/py37/include/z5.

How and where to define manually those 2 paths in order that cmake recognize them?

examples of the related errors:

src/python/lib/graph/rag/CMakeFiles/_rag.dir/build.make:62: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/rag.cxx.o' failed make[2]: *** [src/python/lib/graph/rag/CMakeFiles/_rag.dir/rag.cxx.o] Error 1 In file included from /home/nejat/anaconda3/conda-bld/nifty_1582736589241/_build_env/include/xtensor-python/pytensor.hpp:21:0, from /home/nejat/anaconda3/conda-bld/nifty_1582736589241/work/src/python/lib/graph/rag/project_to_pixels.cxx:6: /home/nejat/anaconda3/conda-bld/nifty_1582736589241/_build_env/include/xtensor-python/pycontainer.hpp:28:10: fatal error: numpy/arrayobject.h: No such file or directory

include "numpy/arrayobject.h"

^~~~~ compilation terminated.

src/python/lib/graph/rag/CMakeFiles/_rag.dir/build.make:134: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/graph_accumulator.cxx.o' failed make[2]: *** [src/python/lib/graph/rag/CMakeFiles/_rag.dir/graph_accumulator.cxx.o] Error 1 In file included from /home/nejat/anaconda3/conda-bld/nifty_1582736589241/work/include/nifty/graph/rag/detail_rag/compute_grid_rag.hxx:22:0, from /home/nejat/anaconda3/conda-bld/nifty_1582736589241/work/include/nifty/graph/rag/grid_rag.hxx:19, from /home/nejat/anaconda3/conda-bld/nifty_1582736589241/work/src/python/lib/graph/rag/accumulate.cxx:5: /home/nejat/anaconda3/conda-bld/nifty_1582736589241/work/include/nifty/z5/z5.hxx:4:10: fatal error: z5/dataset_factory.hxx: No such file or directory

include "z5/dataset_factory.hxx"

^~~~~~~~ compilation terminated. src/python/lib/graph/rag/CMakeFiles/_rag.dir/build.make:182: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/accumulate.cxx.o' failed make[2]: [src/python/lib/graph/rag/CMakeFiles/_rag.dir/accumulate.cxx.o] Error 1 CMakeFiles/Makefile2:1372: recipe for target 'src/python/lib/graph/rag/CMakeFiles/_rag.dir/all' failed make[1]: [src/python/lib/graph/rag/CMakeFiles/_rag.dir/all] Error 2 Copying target nifty to temporary module directory [ 16%] Built target _nifty Makefile:140: recipe for target 'all' failed

constantinpape commented 4 years ago

Apparently, it can not find where numpy and z5 are installed (or, at least their header files). However, the header files of numpy are installed in: /home/nejat/anaconda3/envs/py37/lib/python3.7/site-packages/numpy/core/include/numpy, and those of z5 are installed in /home/nejat/anaconda3/envs/py37/include/z5.

Could you paste the full cmake command you are using please? Also, which OS are you on?

If you want to install from a conda environment, I would recommend to run it like this:

cmake -DCMAKE_PREFIX_PATH=/path/to/conda/env -DNUMPY_INCLUDE_DIR=/path/to/conda/env/lib/python3.7/site-pacakges/numpy/core/include ...

It should pick z5 automatically (if it is installed to this environment) and find numpy by specifying the include dir.

Note that I recently fixed a few minor CMake issues on my fork of nifty, so maybe it would help if you check it out. (I will try to merge this soon, but I am currently updating a few features and want to finish this first.)

arinik9 commented 4 years ago

actually, I am trying to install from a conda environment, mine was a 'conda build' command (I saw it in the README file located at the folder conda-recipe): conda build -c conda-forge --python=3.7 conda-recipe Is not it correct?

What is your full cmake command please ? Thanks!

By the way, I am on linux

constantinpape commented 4 years ago

conda build -c conda-forge --python=3.7 conda-recipe

This is the command to build the conda package, which just wraps cmake; it's easier to use cmake directly.

I just checked the README, and the installation instructions were outdated. Could you try to install with this command:

conda install -c conda-forge -c cpape nifty

Let me know if it does not work, then I will update the instructions on how to build from source and we can try with CMake.

arinik9 commented 4 years ago

thanks for your reply! It did work out with your command. I could be able to import nifty and then do some graph operations. Nevertheless, apparently in this conda package nifty.Configuration.WITH_QPBO is False. Do you have another conda package with WITH_QPBO = True ?

Essentially, since I passed some time with compilation issues, I am interested in compiling from source with CMake. Could you please share with me your Cmake commands ? What I tried is the following:

mkdir bld
cd bld
cmake -DNUMPY_INCLUDE_DIR=/path/to/anaconda3/envs/py37/lib/python3.7/site-pacakges/numpy/core/include -DCMAKE_INSTALL_PREFIX=../nifty-headers -DCMAKE_PREFIX_PATH=/path/to/anaconda3/envs/py37 ..
make install

but it gave me an error related to pybind11, e.g.

In file included from /home/nejat/eclipse/tmp/nifty/src/python/lib/graph/undirected_list_graph.cxx:9:0: /home/nejat/eclipse/tmp/nifty/src/python/lib/graph/export_undirected_graph_class_api.hxx:50:12: error: ‘PYBIND11_DESCR’ does not name a type; did you mean ‘PYBIND11_EVAL’? static PYBIND11_DESCR name() { ^~~~~~ PYBIND11_EVAL In file included from /home/nejat/anaconda3/envs/py37/include/pybind11/attr.h:13:0, from /home/nejat/anaconda3/envs/py37/include/pybind11/pybind11.h:44, from /home/nejat/eclipse/tmp/nifty/src/python/lib/graph/undirected_list_graph.cxx:1:

Thanks in advance!

constantinpape commented 4 years ago

Nevertheless, apparently in this conda package nifty.Configuration.WITH_QPBO is False. Do you have another conda package with WITH_QPBO = True ?

Unfortunately not.

but it gave me an error related to pybind11, e.g.

Can you check which version of pybind11 you are using? I think I have seen this before and fixed it on my branch, can you try building from there?

arinik9 commented 4 years ago

Regarding pybind, its version is 2.4.3

Actually, I was talking about your code. The compilation issue about pybind11 was based on your branch master of https://github.com/constantinpape/nifty

When you have some time, if you can test the installation steps (on a new environment), I would be grateful! Thanks!

constantinpape commented 4 years ago

Regarding pybind, its version is 2.4.3

Actually, I was talking about your code. The compilation issue about pybind11 was based on your branch master of https://github.com/constantinpape/nifty

Ok, I will look into it later.

constantinpape commented 4 years ago

Indeed, the build is broken with pybind11 >= 2.4; see #133. It works with versions < 2.4.

I have also added a section on how to build from source to my branch: https://github.com/constantinpape/nifty#from-source

Hope this helps and let me know if you run into more issues.

arinik9 commented 4 years ago

Thank you very much for you help! I was able to build it with the option qpbo, and import it as package into python.

Nevertheless, I run into some issues with other options, that is, gurobi and lpmp.

In the end, I could be able to compile two versions without problem. But, the problem comes when I import the nifty package. I get the following error in python by typing import nifty:

Traceback (most recent call last): File "", line 1, in File "/opt/anaconda3/envs/GASP/lib/python3.7/site-packages/nifty/init.py", line 4, in from ._nifty import * ImportError: /opt/anaconda3/envs/GASP/lib/python3.7/site-packages/nifty/_nifty.cpython-37m- x86_64-linux-gnu.so: undefined symbol: _ZNK12GRBException10getMessageB5cxx11Ev

I suppose there was something wrong during the compilation. Have you had the same issue before? I mean, would you have an idea what can be the possible reason(s) which allows to compile the code, but we can not use it in execution? Thanks in advance!

constantinpape commented 4 years ago

Thank you very much for you help! I was able to build it with the option qpbo, and import it as package into python.

Glad it works now.

Nevertheless, I run into some issues with other options, that is, gurobi and lpmp.

lpmp support has not been tested in a while and should probably be deprecated. The error you get for gurobi is a linker error. c++filt resolves the symbol to GRBException::getMessage[abi:cxx11]() const. There might be several causes for this; I haven't used the version build with Gurobi, so I don't really know what is going on.

Do you need any Gurobi functionality?

arinik9 commented 4 years ago

Actually, for now I do not need any Gurobi functionality. I just thought that allowing Gurobi would mean to get the exact or LP relaxation solutions, so that would be nice.

constantinpape commented 4 years ago

Actually, for now I do not need any Gurobi functionality. I just thought that allowing Gurobi would mean to get the exact or LP relaxation solutions, so that would be nice.

Yes, that's true. Linker errors can be kinda tricky though, because this might be system dependent.

Anyway, thanks for reporting the issue!