search

pymc-devs / pymc2

THIS IS THE **OLD** PYMC PROJECT (VERSION 2). PLEASE USE PYMC INSTEAD:
http://pymc-devs.github.com/pymc/
Other
879 stars 229 forks source link

pymc fails to find openblas from numpy #190

Closed samfux84 closed 5 years ago

samfux84 commented 5 years ago

In the setup.py file, you are checking for lapack with the command:

lapack_info = get_info('lapack_opt',1)

our numpy is linked against OpenBLAS, which is not detected with this command:

>>> get_info('lapack_opt',1)
/cluster/apps/gdc/python/3.6.1/lib64/python3.6/site-packages/numpy/distutils/system_info.py:624: UserWarning: 
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()
/cluster/apps/gdc/python/3.6.1/lib64/python3.6/site-packages/numpy/distutils/system_info.py:624: UserWarning: 
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  self.calc_info()
/cluster/apps/gdc/python/3.6.1/lib64/python3.6/site-packages/numpy/distutils/system_info.py:624: UserWarning: 
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  self.calc_info()
/cluster/apps/gdc/python/3.6.1/lib64/python3.6/site-packages/numpy/distutils/system_info.py:432: UserWarning: 
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  return cl().get_info(notfound_action)
{}
>>>

on the other hand, numpy.config.show() prints all the necessary information:

>>> numpy.__config__.show()
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']
>>>

What information is required for the installation ?

samfux84 commented 5 years ago

I could solve my problem by removing all

if not lapack_info

sections in setup.py and by replacing extra_info=lapack_info in all add_extension commands with

include_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib'], define_macros = [('HAVE_CBLAS', None)], library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib'], libraries = ['openblas', 'openblas'], runtime_library_dirs = ['/cluster/apps/openblas/0.2.13_seq/x86_64/gcc_4.8.2/lib']

Seems to work:

[sfux@eu-c7-041-01 ~]$ python
Python 3.6.1 (default, Jan 25 2018, 08:42:21) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymc
>>>
fonnesbeck commented 5 years ago

First, are you intending to use PyMC2? We recommend users to work in PyMC3, as it is the current version in development and is far better at this point in time.

If you are interested in PyMC2, please go ahead and submit a PR and I will merge it.

samfux84 commented 5 years ago

I got a request from one of our users to install sonics on our HPC cluster and according

https://github.com/kzkedzierska/sonics

it depends on pymc. I just tried to follow the installation instructions. I don't know if sonics would as well work with PyMC3.

The solution that I implemented is just a manual hack of the setup.py file that is far from a PR that would solve the problem in general.

fonnesbeck commented 5 years ago

Yes, it looks like PyMC2 is what is required, so you have installed the correct version for what your user requires.

I will leave this be for the time being, and see if anyone else reports this issue before trying to dig into it.