Compile failure

[alexbw]

I'm not familiar with the the ins and outs of what weave would do, but my assumption is that this is a system configuration failure.

I'm running on Ryan's neologin cluster, and the HSMM code isn't compiling, which I assume has something to do with missing header files, but I'm not sure.

Just wanted to check if this meant something obvious to you, @mattjj, and if so, what the remedies are. I'm going to check in with the cluster administrators to see if the python-dev package, or equivalent, is installed.

Traceback (most recent call last):
  File "correctness_tests/real_data_likelihoods.py", line 70, in <module>
    model.resample_model()
  File "/scratch/global/alexbw/Code/pyhsmm/models.py", line 465, in resample_model
    super(HSMM,self).resample_model(**kwargs)
  File "/scratch/global/alexbw/Code/pyhsmm/models.py", line 152, in resample_model
    self.resample_obs_distns()
  File "/share/isilon-scratch/alexbw/Code/pyhsmm_library_models/library_models.py", line 280, in resample_obs_distns
    **kwargs)
  File "/share/isilon-scratch/alexbw/Code/pyhsmm_library_models/library_models.py", line 73, in resample_from_likelihoods
    z = sample_discrete_from_log_2d_destructive(scores)
  File "/scratch/global/alexbw/Code/pyhsmm/util/stats.py", line 111, in sample_discrete_from_log_2d_destructive
    extra_compile_args=['-O3','-DNDEBUG'])
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/inline_tools.py", line 357, in inline
    **kw)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/inline_tools.py", line 484, in compile_function
    verbose=verbose, **kw)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/ext_tools.py", line 369, in compile
    verbose = verbose, **kw)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/build_tools.py", line 273, in build_extension
    setup(name = module_name, ext_modules = [ext],verbose=verb)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/distutils/core.py", line 186, in setup
    return old_setup(**new_attr)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/distutils/core.py", line 169, in setup
    raise SystemExit, "error: " + str(msg)
scipy.weave.build_tools.CompileError: error: Command "g++ -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/scratch/global/alexbw/Code/pyhsmm/util/../deps/Eigen3 -I/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave -I/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/scxx -I/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include -I/scratch/global/alexbw/anaconda/include/python2.7 -c /home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4551.cpp -o /tmp/alexbw/python27_intermediate/compiler_9670339c2c408eb4b226bf1cdd124510/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4551.o -O3 -DNDEBUG" failed with exit status 1

[mattjj]

The compiler error which caused that should have printed above what you pasted (before the Python stack trace). Can you paste that?

[alexbw]

In file included from /scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/
numpy/ndarraytypes.h:1728,
                 from /scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/
numpy/ndarrayobject.h:17,
                 from /scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/
numpy/arrayobject.h:15,
                 from /home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:22:
/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_a
pi.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_A
PI NPY_1_7_API_VERSION"
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:23:22: error: Eigen/Core: No
 such file or directory
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp: In function ‘PyObject* comp
iled_func(PyObject*, PyObject*)’:
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:700: error: ‘Eigen’ is not a
 namespace-name
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:700: error: expected namespa
ce-name before ‘;’ token
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:703: error: ‘Map’ was not de
clared in this scope
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:703: error: ‘ArrayXd’ was no
t declared in this scope
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:703: error: ‘vals’ was not d
eclared in this scope
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:678: warning: unused variabl
e ‘Nscores’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:679: warning: unused variabl
e ‘Sscores’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:680: warning: unused variabl
e ‘Dscores’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:693: warning: unused variabl
e ‘Nout’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:694: warning: unused variabl
e ‘Sout’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:695: warning: unused variabl
e ‘Dout’
In file included from /scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/
numpy/ndarraytypes.h:1728,
                 from /scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/[19/583]
numpy/ndarrayobject.h:17,
                 from /scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/
numpy/arrayobject.h:15,
                 from /home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:22:
/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_a
pi.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_A
PI NPY_1_7_API_VERSION"
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:23:22: error: Eigen/Core: No
 such file or directory
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp: In function ‘PyObject* comp
iled_func(PyObject*, PyObject*)’:
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:700: error: ‘Eigen’ is not a
 namespace-name
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:700: error: expected namespa
ce-name before ‘;’ token
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:703: error: ‘Map’ was not de
clared in this scope
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:703: error: ‘ArrayXd’ was no
t declared in this scope
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:703: error: ‘vals’ was not d
eclared in this scope
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:678: warning: unused variabl
e ‘Nscores’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:679: warning: unused variabl
e ‘Sscores’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:680: warning: unused variabl
e ‘Dscores’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:693: warning: unused variabl
e ‘Nout’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:694: warning: unused variabl
e ‘Sout’
/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.cpp:695: warning: unused variabl
e ‘Dout’
Traceback (most recent call last):
  File "correctness_tests/real_data_likelihoods.py", line 70, in <module>
    model.resample_model()
  File "/scratch/global/alexbw/Code/pyhsmm/models.py", line 465, in resample_model
    super(HSMM,self).resample_model(**kwargs)
  File "/scratch/global/alexbw/Code/pyhsmm/models.py", line 152, in resample_model
    self.resample_obs_distns()
  File "/share/isilon-scratch/alexbw/Code/pyhsmm_library_models/library_models.py", line 280, in resa
mple_obs_distns
    **kwargs)
  File "/share/isilon-scratch/alexbw/Code/pyhsmm_library_models/library_models.py", line 73, in resam
ple_from_likelihoods
    z = sample_discrete_from_log_2d_destructive(scores)
  File "/scratch/global/alexbw/Code/pyhsmm/util/stats.py", line 111, in sample_discrete_from_log_2d_d
estructive
    extra_compile_args=['-O3','-DNDEBUG'])
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/inline_tools.py", lin
e 357, in inline
    **kw)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/inline_tools.py", lin
e 484, in compile_function
    verbose=verbose, **kw)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/ext_tools.py", line 3
69, in compile
    verbose = verbose, **kw)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave/build_tools.py", line
 273, in build_extension
    setup(name = module_name, ext_modules = [ext],verbose=verb)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/numpy/distutils/core.py", line 18
6, in setup
    return old_setup(**new_attr)
  File "/scratch/global/alexbw/anaconda/lib/python2.7/distutils/core.py", line 169, in setup
    raise SystemExit, "error: " + str(msg)
scipy.weave.build_tools.CompileError: error: Command "g++ -pthread -fno-strict-aliasing -g -O2 -DNDEB
UG -g -fwrapv -O3 -Wall -fPIC -I/scratch/global/alexbw/Code/pyhsmm/util/../deps/Eigen3 -I/scratch/glo
bal/alexbw/anaconda/lib/python2.7/site-packages/scipy/weave -I/scratch/global/alexbw/anaconda/lib/pyt
hon2.7/site-packages/scipy/weave/scxx -I/scratch/global/alexbw/anaconda/lib/python2.7/site-packages/n
umpy/core/include -I/scratch/global/alexbw/anaconda/include/python2.7 -c /home/alexbw/.python27_compi
led/sc_61dd1b3698147cae1ceb0083a14da4552.cpp -o /tmp/alexbw/python27_intermediate/compiler_9670339c2c
408eb4b226bf1cdd124510/home/alexbw/.python27_compiled/sc_61dd1b3698147cae1ceb0083a14da4552.o -O3 -DND
EBUG" failed with exit status 1

[mattjj]

error: Eigen/Core: No such file or directory

It's not able to include the Eigen headers, though they're being passed to the compile line via

-I/scratch/global/alexbw/Code/pyhsmm/util/../deps/Eigen3

Is scratch a network drive? If so, you should check if the Eigen submodule was properly cloned via --recursive (if not, a git submodule update --init --recursive would fix that). If it's actually a local drive, maybe the repo isn't being cloned locally properly.

[mattjj]

It's not a cluster config error that the admins would need to deal with, though. It's just about Eigen includes.

[alexbw]

I missed that line! I'll check it out

On Sat, Aug 10, 2013 at 1:28 PM, Matthew Johnson notifications@github.comwrote:

It's not a cluster config error that the admins would need to deal with, though. It's just about Eigen includes.

— Reply to this email directly or view it on GitHubhttps://github.com/dattalab/pyhsmm-library-models/issues/22#issuecomment-22443673 .

[alexbw]

It works because duh I don't understand git duh duh

On Sat, Aug 10, 2013 at 2:29 PM, Alex Wiltschko alex.bw@gmail.com wrote:

I missed that line! I'll check it out

On Sat, Aug 10, 2013 at 1:28 PM, Matthew Johnson <notifications@github.com

wrote:

It's not a cluster config error that the admins would need to deal with, though. It's just about Eigen includes.

— Reply to this email directly or view it on GitHubhttps://github.com/dattalab/pyhsmm-library-models/issues/22#issuecomment-22443673 .