PythonMars fails to build

[AlexanderFabisch]

Seems like some dependency is missing. What should I install?

 [~/infuse/simulation/mars/plugins/PythonMars/build]  (master)
$ make
[ 20%] Automatic moc for target PythonMars
[ 20%] Built target PythonMars_automoc
[ 40%] Building CXX object CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o
/home/dfki.uni-bremen.de/afabisch/infuse/simulation/mars/plugins/PythonMars/src/PythonInterpreter.cpp:8:31: fatal error: numpy/arrayobject.h: No such file or directory
compilation terminated.
CMakeFiles/PythonMars.dir/build.make:86: recipe for target 'CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o' failed
make[2]: *** [CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/PythonMars.dir/all' failed
make[1]: *** [CMakeFiles/PythonMars.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

[AlCap23]

Hi @AlexanderFabisch ,

I recently encountered the same problem. Looks like numpy is not found. I suspect you are using conda or another kind of virutal environment to manage python?

With the help of @malter we were able to install mars with first activating a suitable ( for me python 2.7 worked fine, python >= 3. is not supported right now ) environment before running the install script. If you are working on Linux in /simulation/mars/plugins/PythonMars/src/PythonMars.cpp Line 53 the python library had to be replaced from

dlopen("libpython2.7.so.1", RTLD_LAZY | RTLD_GLOBAL);

with

dlopen("libpython2.7.so", RTLD_LAZY | RTLD_GLOBAL);

Which is just the name of the corresponding pythonlib in the virtual environment.

Additionally add the path to your python environment to the env.sh, Line 5, e.g.

export LD_LIBRARY_PATH="/foo/miniconda/envs/py27/lib:$LD_LIBRARY_PATH"

There may be build errors regarding ODE as well. We were able to work around them by building ODE by hand without additional arguments first.

Hope that helps.

[AlexanderFabisch]

Hi,

I usually use the standard Python from Ubuntu. I didn't even want to build PythonMars though. Never had any such problems before although I work quite often on interfaces between C++ and Python, also with NumPy.

[malter]

Hey,

first of all, if there is a problem with a plugin it is not loaded and if you don't use it everything should work fine for your application. But of course we should solve the problems and currently it looks like python problems are arising more and more. ;-) The standard Python from Ubuntu didn't made any problems so far. What is the Ubuntu version that you are using? I will try to reproduce your problem to resolve it as soon as possible.

[AlexanderFabisch]

16.04, maybe too old?

[AlexanderFabisch]

btw. I used autoproj installation for installation and this buildconf: ... gitlab ... InFuse/buildconf/tree/dfki.seekurjr

[ChristophPetzoldt]

Hey, I just encountered the same problem when tying to build PythonMars:

[~/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build] (master) $ make [ 20%] Automatic moc for target PythonMars [ 20%] Built target PythonMars_automoc [ 40%] Building CXX object CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonInterpreter.cpp:8:31: fatal error: numpy/arrayobject.h: Datei oder Verzeichnis nicht gefunden compilation terminated. CMakeFiles/PythonMars.dir/build.make:86: die Regel für Ziel „CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o“ scheiterte make[2]: [CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o] Fehler 1 CMakeFiles/Makefile2:67: die Regel für Ziel „CMakeFiles/PythonMars.dir/all“ scheiterte make[1]: [CMakeFiles/PythonMars.dir/all] Fehler 2 Makefile:127: die Regel für Ziel „all“ scheiterte make: *** [all] Fehler 2

Same as Alex, I'm also using the standard Python from Ubuntu (V2.7.12) and Ubuntu 16.04.

[malter]

I was able to build MARS without any adaptations on a clean installed Ubuntu 16.04 and 18.04. So I guess the error have to be related to your system configuration. There must be a conflict with another python installation / package. Which version is the default if you type "python"? Does import numpy works in your default installation. What is the output of "cd simulation/plugins/PyhtonMars/build; make VERBOSE=1"?

[ChristophPetzoldt]

$ python Python 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information.

>>> import numpy
>>> 

$ cd simulation/mars/plugins/PythonMars/build/ $ make VERBOSE=1 /usr/bin/cmake -H/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars -B/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake -E cmake_progress_start /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build/CMakeFiles /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build/CMakeFiles/progress.marks make -f CMakeFiles/Makefile2 all make[1]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird betreten make -f CMakeFiles/PythonMars_automoc.dir/build.make CMakeFiles/PythonMars_automoc.dir/depend make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird betreten cd /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build/CMakeFiles/PythonMars_automoc.dir/DependInfo.cmake --color= make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird verlassen make -f CMakeFiles/PythonMars_automoc.dir/build.make CMakeFiles/PythonMars_automoc.dir/build make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird betreten [ 20%] Automatic moc for target PythonMars /usr/bin/cmake -E cmake_autogen /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build/CMakeFiles/PythonMars_automoc.dir/ DEBUG AUTOGEN: Checking /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonMars.cpp AUTOGEN: Checking /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonInterpreter.cpp AUTOGEN: Checking /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonInterpreter.hpp AUTOGEN: Checking /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonMars.h make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird verlassen [ 20%] Built target PythonMars_automoc make -f CMakeFiles/PythonMars.dir/build.make CMakeFiles/PythonMars.dir/depend make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird betreten cd /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build/CMakeFiles/PythonMars.dir/DependInfo.cmake --color= make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird verlassen make -f CMakeFiles/PythonMars.dir/build.make CMakeFiles/PythonMars.dir/build make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird betreten [ 40%] Building CXX object CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o /usr/bin/c++ -DGIT_INFO -DGIT_INFO_REV=c1824025e6be4320f96637c2155227a41998765e -DGIT_INFO_SRC=\"autobuildhttps://github.com/rock-simulation/mars\" -DHAVE_OSG_VERSION_H -DPYTHON_SUPPORT -DPythonMars_EXPORTS -DQT_CORE_LIB -DQT_GUI_LIB -DQT_WIDGETS_LIB -DUSE_MEMORYVIEWS -DUSE_TR1 -I/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build -I/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars -I/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/install/include -I/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/install/include/base/backward -I/usr/include/eigen3 -I/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src -I/usr/include/python2.7 -isystem /usr/include/x86_64-linux-gnu/qt5 -isystem /usr/include/x86_64-linux-gnu/qt5/QtWidgets -isystem /usr/include/x86_64-linux-gnu/qt5/QtGui -isystem /usr/include/x86_64-linux-gnu/qt5/QtCore -isystem /usr/lib/x86_64-linux-gnu/qt5/mkspecs/linux-g++-64 -fPIC -std=c++11 -Wall -g -fPIC -fPIC -o CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o -c /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonInterpreter.cpp /home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonInterpreter.cpp:8:31: fatal error: numpy/arrayobject.h: Datei oder Verzeichnis nicht gefunden compilation terminated. CMakeFiles/PythonMars.dir/build.make:86: die Regel für Ziel „CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o“ scheiterte make[2]: [CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o] Fehler 1 make[2]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird verlassen CMakeFiles/Makefile2:67: die Regel für Ziel „CMakeFiles/PythonMars.dir/all“ scheiterte make[1]: [CMakeFiles/PythonMars.dir/all] Fehler 2 make[1]: Verzeichnis „/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/build“ wird verlassen Makefile:127: die Regel für Ziel „all“ scheiterte make: *** [all] Fehler 2

[malter]

after "import numpy" you may can print it "print numpy" that should give you the place where it is loaded

[ChristophPetzoldt]

>>> import numpy
>>> print numpy
<module 'numpy' from '/home/dfki.uni-bremen.de/cpetzoldt/.local/lib/python2.7/site-packages/numpy/__init__.pyc'>

[malter]

It looks like numpy isn't installed in your system. How have you installed numpy? Can you try to reinstall it with apt-get (python-numpy)?

[ChristophPetzoldt]

Just uninstalled numpy using pip uninstall numpy and installed it using sudo apt-get install python-numpy.

Python is loaded from usr/local now:

$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy 
>>> print numpy
<module 'numpy' from '/usr/local/lib/python2.7/dist-packages/numpy/__init__.pyc'>

However, when trying to build PythonMars, I still get the same error (also after removing the PythonMars build-folder and re-running bob-install in the autoproj folder, I still get the same result):

$ make install
[ 20%] Automatic moc for target PythonMars
[ 20%] Built target PythonMars_automoc
[ 40%] Building CXX object CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o
/home/dfki.uni-bremen.de/cpetzoldt/Dokumente/dev_root/mars_dev/simulation/mars/plugins/PythonMars/src/PythonInterpreter.cpp:8:31: fatal error: numpy/arrayobject.h: Datei oder Verzeichnis nicht gefunden
compilation terminated.
CMakeFiles/PythonMars.dir/build.make:86: die Regel für Ziel „CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o“ scheiterte
make[2]: *** [CMakeFiles/PythonMars.dir/src/PythonInterpreter.cpp.o] Fehler 1
CMakeFiles/Makefile2:67: die Regel für Ziel „CMakeFiles/PythonMars.dir/all“ scheiterte
make[1]: *** [CMakeFiles/PythonMars.dir/all] Fehler 2
Makefile:127: die Regel für Ziel „all“ scheiterte
make: *** [all] Fehler 2

respectively:

Errors:
 - configure: tools/logger
 - build: simulation/smurf_parser
 - build: simulation/mars/common/gui/data_broker_plotter2
 - build: simulation/mars/plugins/PythonMars

[malter]

Now you should have a numpy folder in "/usr/include/python2.7". If so, it is not clear to me why it doesn't find the headers. If not, I don't know why it is not there. ;-)

bob-showlog smurf_parser?

[ChristophPetzoldt]

That was the problem, thank you! The numpy symlink in usr/include/python2.7 was pointing to the wrong location (/usr/lib/python2.7/dist-packages/numpy/core/include/numpy). I changed the symlink to point to /usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy. Building PythonMars works now! Thanks again! :+1: