search

INCF / MUSIC

MUSIC, the MUltiSimulation Coordinator
GNU General Public License v3.0
37 stars 37 forks source link

Error when compiling #1

Closed MartyMcFlyHasNoFarmHiaHiaHo closed 8 years ago

MartyMcFlyHasNoFarmHiaHiaHo commented 8 years ago

Hi,

I get the following issue when compiling MUSIC

Makefile:336: warning: ignoring old commands for target predict_rank.py' make[2]: Nothing to be done forall'. make[2]: Leaving directory /home/fzi/Packages/MUSIC/music-config' Making all in pymusic make[2]: Entering directory/home/fzi/Packages/MUSIC/pymusic' cd ../pymusic; /usr/bin/python -c \ "from Cython.Build import cythonize; \ cythonize('pymusic.pyx', verbose=1)" cd ../pymusic; /usr/bin/python -c \ "from Cython.Build import cythonize; \ cythonize('pybuffer.pyx', verbose=1)" missing cimport in module 'mpi4py.mpi_c': pybuffer.pxd missing cimport in module 'mpi4py.mpi_c': pymusic.pxd Compiling pybuffer.pyx because it changed. [1/1] Cythonizing pybuffer.pyx Compiling /home/fzi/Packages/MUSIC/pymusic/pybuffer.pyx missing cimport in module 'mpi4py.mpi_c': ./music/pybuffer.pxd Compiling pymusic.pyx because it changed. [1/1] Cythonizing pymusic.pyx Compiling /home/fzi/Packages/MUSIC/pymusic/pymusic.pyx

Error compiling Cython file:

...

which is lacking for a certain combination of MPI and mpi4py versions

cdef extern from "mpi_compat.h": pass

cimport mpi4py.MPI as MPI from mpi4py.mpi_c cimport *

^

pybuffer.pxd:7:0: 'mpi4py/mpi_c.pxd' not found

Error compiling Cython file:

...

which is lacking for a certain combination of MPI and mpi4py versions

cdef extern from "mpi_compat.h": pass

cimport mpi4py.MPI as MPI from mpi4py.mpi_c cimport *

^

pymusic.pxd:7:0: 'mpi4py/mpi_c.pxd' not found Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 877, in cythonize cythonize_one(_args) File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 997, in cythonize_one raise CompileError(None, pyxfile) Cython.Compiler.Errors.CompileError: pybuffer.pyx make[2]: ** [pybuffer.cpp] Error 1 make[2]: *\ Waiting for unfinished jobs....

Error compiling Cython file:

... void tick()

cdef extern void cython_callback(PyObject*, double, IndexType, int)

cdef extern from "music/music_c.h" namespace "MUSIC": cdef inline MPI_Comm communicator(CSetup*)

^

pymusic.pxd:136:16: 'MPI_Comm' is not a type identifier

Error compiling Cython file:

...

cdef extern void cython_callback(PyObject*, double, IndexType, int)

cdef extern from "music/music_c.h" namespace "MUSIC": cdef inline MPIComm communicator(CSetup) cdef inline MPIComm communicator(CRuntime)

^

pymusic.pxd:137:16: 'MPI_Comm' is not a type identifier

Error compiling Cython file:

... cdef Args r = argv_toc(argv if argv is not None else sys.argv) try: if required is None: self.ptr = new CSetup(r.argc, r.argv) self.provided = MPI_THREAD_SINGLE

^

pymusic.pyx:377:49: undeclared name not builtin: MPI_THREAD_SINGLE

Error compiling Cython file:

... index (0): if index_map is None, this is the base offset for constructing a linear index """ self.buf = buf self.ptr = new CArrayData(buf.pybuf.buf, buf.dtype.ob_mpi,

^

pymusic.pyx:601:43: Cannot convert 'MPI_Datatype' to Python object

Error compiling Cython file:

... self.ptr = new CArrayData(buf.pybuf.buf, buf.dtype.ob_mpi, index, buf.items) \ if index_map is None else \ new CArrayData(buf.pybuf.buf, buf.dtype.ob_mpi,

^

pymusic.pyx:605:43: Cannot convert 'MPI_Datatype' to Python object Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 877, in cythonize cythonize_one(_args) File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 997, in cythonize_one raise CompileError(None, pyxfile) Cython.Compiler.Errors.CompileError: pymusic.pyx make[2]: ** [pymusic.cpp] Error 1 make[2]: Leaving directory /home/fzi/Packages/MUSIC/pymusic' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory/home/fzi/Packages/MUSIC' make: *\ [all] Error 2

I am not sure if this depends on my MPI version but mpi4py runs without issues

mdjurfeldt commented 8 years ago

Which versions of cython, MPI and mpi4py do you use?

MartyMcFlyHasNoFarmHiaHiaHo commented 8 years ago

Cython: 0.23.4 MPI: OpenMPI 1.10.1 Mpi4Py: 2.0.0 Python 2.7 (virtualenv)

I also tried OpenMPI 1.6.5 (and open Mpi4py compiled against that)

The same problem occurs with both openMPI versions, I was able to reproduce the error on an Ubuntu 14.04 docker image on another machine (with openmpi 1.6.5)

apeyser commented 8 years ago

You have a much newer setup than I do, so the versions should be any problem. Somehow the path for mpi4py is being lost. Did you use the virtualenv python to install mpi4py? I see that the cython path is a /usr/local path -- is the entire toolchain installed to /usr/local?

apeyser commented 8 years ago

You can try dropping in the cythonized outputs: https://gist.github.com/apeyser/49986addf1a5a7d1c07b which are pymusic/pymusic.cpp and pymusic/pybuffer.cpp to see whether the rest of your environment works.

uahic commented 8 years ago

It seems that mpi4py 2.0.0 is the issue;

Errorin pybuffer.pxd: pybuffer.pxd:7:0: 'mpi4py/mpi_c.pxd' not found

which I think does not exist anymore in the most recent version

JanneM commented 8 years ago

uahic, I opened an issue with a patch about that a couple of days ago. Basically, you need to use "libmpi" instead of "mpi_c" with version 2.

mdjurfeldt commented 8 years ago

Merged now! Thank you Janne!