Can't compile on OS X 10.9.3

[houbysoft]

Hi,

I'm running into a similar issue as https://github.com/mattjj/pyhsmm/issues/21. I'm on OS X 10.9.3, and I'm using clang (Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)).

I tried building in two ways specified by the README but both failed for me.

First I tried:

$ python setup.py build_ext --inplace running build_ext gcc-4.2 not found, using clang instead building 'internals.hmm_messages_interface' extension clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP In file included from internals/hmm_messages_interface.cpp:360: In file included from internals/hmm_messages.h:4: In file included from deps/Eigen3/Eigen/Core:287: deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p;

                 log10
... bunch of other errors follow ...

I also tried the suggested alternative for clang but got:
$ python setup.py build_ext --inplace --with-old-clang
running build_ext
gcc-4.2 not found, using clang instead
building 'internals.hmm_messages_interface' extension
clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP -stdlib=libc++
clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later)
clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later)
clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later)
error: command 'clang' failed with exit status 1

Any ideas?

Thanks

[houbysoft]

Update: I found how to force it to build for 10.7 or later by adding

extra_compile_args.append('-mmacosx-version-min=10.7') extra_link_args.append('-mmacosx-version-min=10.7')

to setup.py.

However now it gives these errors: running build_ext gcc-4.2 not found, using clang instead building 'internals.hmm_messages_interface' extension clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP -stdlib=libc++ -mmacosx-version-min=10.7 internals/hmm_messagesinterface.cpp:10976:604: error: cannot initialize a parameter of type 'int ' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(___Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:11254:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:11532:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:11810:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:12088:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:12366:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:12644:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:12922:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:13200:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:13478:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:13756:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:14034:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:14312:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:14590:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:14868:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:15146:612: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:44750:385: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_9, pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:44993:389: error: cannot initialize a parameter of type 'int ' with an rvalue of type '__pyx_t_5numpy_int32_t ' (aka 'long *') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType randseq) ^ internals/hmm_messages_interface.cpp:45236:385: error: cannot initialize a parameter of type 'int *' with an rvalue of type 'pyx_t_5numpy_int32_t ' (aka 'long ') ...(&(_Pyx_BufPtrCContig1d(pyx_t_5numpy_int32_t , pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))... ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here IntType stateseq, FloatType *randseq) ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'clang' failed with exit status 1

[mattjj]

The build directory still has 10.6 in the name. Did you clean everything before trying to rebuild?

[houbysoft]

Hmm even if I delete the build dir and rerun it the directory still has 10.6 in the name, not sure how to change that / where it's getting that from?

[mattjj]

I don't know either.

I'm sure this is fixable, but it might be a lot easier if you just install a recent gcc instead. Not only would the build be easier, but gcc generates slightly better code for this numerical stuff, especially for recent Intel chips.

[houbysoft]

Alright, thanks for the quick responses! I'll try it with gcc

[mattjj]

Good luck!

[houbysoft]

I just tried it with gcc, but sadly still no luck. I installed the gcc from homebrew (brew install gcc), which is currently version 4.8.

I then modified setup.py a bit so it can find it: import os os.environ["CC"] = "gcc-4.8" os.environ["CXX"] = "g++-4.8"

However, when I run python setup.py build_ext --inplace I still get some errors, this time about the assembly it produces it seems. I'll continue trying to find a solution but if you have any ideas it'd be great.

$ python setup.py build_ext --inplace running build_ext building 'internals.hmm_messages_interface' extension gcc-4.8 -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:13:bad register name %rdi)' /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:14:bad register name%rax)' /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:21:bad register name %rdi)' /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:36:bad register name%rdi)' /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:40:bad register name %rdi)' /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:41:bad register name%rax)' /var/folders/g4/b74fdtg56_jc_v_5swn3dfym0000gn/T//cc5WHPPp.s:65:bad register name `%rdi)' ... bunch of similar errors ...

[mattjj]

It looks like your CFLAGS is passing both -arch i386 and -arch x86_64, which is causing bad register name errors. You should figure out where those flags are coming from (they might be from your shell environment).

This problem doesn't look specific to this code. Can you build any cython code on your system?

[mattjj]

Just so you know, the lines should look something like this:

building 'internals.hmm_messages_interface' extension
/opt/local/bin//g++ -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ideps/Eigen3/ -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.8-x86_64-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP

The only flags pyhsmm adds are on the end (namely -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP). The rest come from cython (which specifies the python framework directories, from macports in this case, and the other flags) and any environment variables like CFLAGS (mine is empty).

[nfoti]

I've gotten pyhsmm to build on a fresh install of Mavericks with gcc-4.8. For whatever reason the Cython in Anaconda wants to use the GNU OpenMP library which Apple doesn't include in clang anymore. So gcc is the only way to go now.

I found that with Mountain Lion I had to explicitly set the ARCH_FLAGS environment variable so that the 32-bit code wasn't built (see here). It has something to do with universal versions of libraries being used, but I'm not sure why they'd be showing up. Anyways, it should work once you get the ARCH_FLAGS right.

[houbysoft]

Awesome, the link @nfoti provided got it to build. For anyone facing the same issues, the following setup.py ended up working for me.

from distutils.core import setup
import numpy as np
import sys
import os

from util.cyutil import cythonize # my version of Cython.Build.cythonize

# NOTE: distutils makes no sense

os.environ["CC"] = "gcc-4.8"
os.environ["CXX"] = "g++-4.8"
os.environ["ARCHFLAGS"] = "-arch x86_64"

extra_link_args = []
extra_compile_args = ['-DHMM_TEMPS_ON_HEAP']

if '--with-old-clang' in sys.argv:
    sys.argv.remove('--with-old-clang')
    extra_compile_args.append('-stdlib=libc++')
    extra_link_args.append('-stdlib=libc++')

if '--with-openmp' in sys.argv:
    sys.argv.remove('--with-openmp')
    extra_compile_args.append('-fopenmp')
    extra_link_args.append('-fopenmp')

if '--with-native' in sys.argv:
    sys.argv.remove('--with-native')
    extra_compile_args.append('-march=native')

ext_modules = cythonize('**/*.pyx')
for e in ext_modules:
    e.extra_compile_args.extend(extra_compile_args)
    e.extra_link_args.extend(extra_link_args)

setup(
    ext_modules=ext_modules,
    include_dirs=[np.get_include(),],
)