Is there any environment variable needed, apart from the BUILDPACK one?

ChristopherRabotin commented 10 years ago

My requirements include the following (in that order):

Cython==0.20.2 # Compiles fine.
numpy==1.8.1
# ...
scipy==0.14.0

However, on deploy, I get the following error (after Cython compiles correctly, and numpy starts to compile correctly). Any thoughts on the reasons why this can happen?

gcc: numpy/random/mtrand/randomkit.c
           gcc -pthread -shared build/temp.linux-x86_64-2.7/numpy/random/mtrand/mtrand.o build/temp.linux-x86_64-2.7/numpy/random/mtrand/randomkit.o build/temp.linux-x86_64-2.7/numpy/random/mtrand/initarray.o build/temp.linux-x86_64-2.7/numpy/random/mtrand/distributions.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/random/mtrand.so
           Creating build/scripts.linux-x86_64-2.7/f2py
             adding 'build/scripts.linux-x86_64-2.7/f2py' to scripts
           changing mode of build/scripts.linux-x86_64-2.7/f2py from 600 to 755

           warning: no files found matching 'tools/py3tool.py'
           warning: no files found matching '*' under directory 'doc/f2py'
           changing mode of /app/.heroku/python/bin/f2py to 755
         Running setup.py install for scipy
           blas_opt_info:
           blas_mkl_info:
             libraries mkl,vml,guide not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']
             NOT AVAILABLE

           openblas_info:
             libraries  not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']
             NOT AVAILABLE

           atlas_blas_threads_info:
           Setting PTATLAS=ATLAS
             libraries ptf77blas,ptcblas,atlas not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']
             NOT AVAILABLE

           atlas_blas_info:
             libraries f77blas,cblas,atlas not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']
             NOT AVAILABLE

           /app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py:1521: UserWarning:
               Atlas (http://math-atlas.sourceforge.net/) libraries not found.
               Directories to search for the libraries can be specified in the
               numpy/distutils/site.cfg file (section [atlas]) or by setting
               the ATLAS environment variable.
             warnings.warn(AtlasNotFoundError.__doc__)
           blas_info:
             libraries blas not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']
             NOT AVAILABLE

           /app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py:1530: UserWarning:
               Blas (http://www.netlib.org/blas/) libraries not found.
               Directories to search for the libraries can be specified in the
               numpy/distutils/site.cfg file (section [blas]) or by setting
               the BLAS environment variable.
             warnings.warn(BlasNotFoundError.__doc__)
           blas_src_info:
             NOT AVAILABLE

           /app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py:1533: UserWarning:
               Blas (http://www.netlib.org/blas/) sources not found.
               Directories to search for the sources can be specified in the
               numpy/distutils/site.cfg file (section [blas_src]) or by setting
               the BLAS_SRC environment variable.
             warnings.warn(BlasSrcNotFoundError.__doc__)
           Traceback (most recent call last):
             File "<string>", line 1, in <module>
             File "/tmp/pip-build-u6095/scipy/setup.py", line 237, in <module>
               setup_package()
             File "/tmp/pip-build-u6095/scipy/setup.py", line 234, in setup_package
               setup(**metadata)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/core.py", line 135, in setup
               config = configuration()
             File "/tmp/pip-build-u6095/scipy/setup.py", line 173, in configuration
               config.add_subpackage('scipy')
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 966, in add_subpackage
               caller_level = 2)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 935, in get_subpackage
               caller_level = caller_level + 1)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 872, in _get_configuration_from_setup_py
               config = setup_module.configuration(*args)
             File "scipy/setup.py", line 12, in configuration
               config.add_subpackage('integrate')
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 966, in add_subpackage
               caller_level = 2)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 935, in get_subpackage
               caller_level = caller_level + 1)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 872, in _get_configuration_from_setup_py
               config = setup_module.configuration(*args)
             File "scipy/integrate/setup.py", line 12, in configuration
               blas_opt = get_info('blas_opt',notfound_action=2)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 350, in get_info
               return cl().get_info(notfound_action)
             File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 509, in get_info
               raise self.notfounderror(self.notfounderror.__doc__)
           numpy.distutils.system_info.BlasNotFoundError:
               Blas (http://www.netlib.org/blas/) libraries not found.
               Directories to search for the libraries can be specified in the
               numpy/distutils/site.cfg file (section [blas]) or by setting
               the BLAS environment variable.
           Complete output from command /app/.heroku/python/bin/python -c "import setuptools;__file__='/tmp/pip-build-u6095/scipy/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-t2O5oi-record/install-record.txt --single-version-externally-managed:
           blas_opt_info:

       blas_mkl_info:

         libraries mkl,vml,guide not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']

         NOT AVAILABLE

       openblas_info:

         libraries  not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']

         NOT AVAILABLE

       atlas_blas_threads_info:

       Setting PTATLAS=ATLAS

         libraries ptf77blas,ptcblas,atlas not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']

         NOT AVAILABLE

       atlas_blas_info:

         libraries f77blas,cblas,atlas not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']

         NOT AVAILABLE

       /app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py:1521: UserWarning:

           Atlas (http://math-atlas.sourceforge.net/) libraries not found.

           Directories to search for the libraries can be specified in the

           numpy/distutils/site.cfg file (section [atlas]) or by setting

           the ATLAS environment variable.

         warnings.warn(AtlasNotFoundError.__doc__)

       blas_info:

         libraries blas not found in ['/app/.heroku/python/lib', '/usr/local/lib', '/usr/lib64', '/usr/lib']

         NOT AVAILABLE

       /app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py:1530: UserWarning:

           Blas (http://www.netlib.org/blas/) libraries not found.

           Directories to search for the libraries can be specified in the

           numpy/distutils/site.cfg file (section [blas]) or by setting

           the BLAS environment variable.

         warnings.warn(BlasNotFoundError.__doc__)

       blas_src_info:

         NOT AVAILABLE

       /app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py:1533: UserWarning:

           Blas (http://www.netlib.org/blas/) sources not found.

           Directories to search for the sources can be specified in the

           numpy/distutils/site.cfg file (section [blas_src]) or by setting

           the BLAS_SRC environment variable.

         warnings.warn(BlasSrcNotFoundError.__doc__)

       Traceback (most recent call last):

         File "<string>", line 1, in <module>

         File "/tmp/pip-build-u6095/scipy/setup.py", line 237, in <module>

           setup_package()

         File "/tmp/pip-build-u6095/scipy/setup.py", line 234, in setup_package

           setup(**metadata)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/core.py", line 135, in setup

           config = configuration()

         File "/tmp/pip-build-u6095/scipy/setup.py", line 173, in configuration

           config.add_subpackage('scipy')

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 966, in add_subpackage

           caller_level = 2)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 935, in get_subpackage

           caller_level = caller_level + 1)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 872, in _get_configuration_from_setup_py

           config = setup_module.configuration(*args)

         File "scipy/setup.py", line 12, in configuration

           config.add_subpackage('integrate')

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 966, in add_subpackage

           caller_level = 2)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 935, in get_subpackage

           caller_level = caller_level + 1)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 872, in _get_configuration_from_setup_py

           config = setup_module.configuration(*args)

         File "scipy/integrate/setup.py", line 12, in configuration

           blas_opt = get_info('blas_opt',notfound_action=2)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 350, in get_info

           return cl().get_info(notfound_action)

         File "/app/.heroku/python/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 509, in get_info

           raise self.notfounderror(self.notfounderror.__doc__)

       numpy.distutils.system_info.BlasNotFoundError:

           Blas (http://www.netlib.org/blas/) libraries not found.

           Directories to search for the libraries can be specified in the

           numpy/distutils/site.cfg file (section [blas]) or by setting

           the BLAS environment variable.

       ----------------------------------------
       Command /app/.heroku/python/bin/python -c "import setuptools;__file__='/tmp/pip-build-u6095/scipy/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-t2O5oi-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-u6095/scipy
       Storing complete log in /app/.pip/pip.log

 !     Push rejected, failed to compile Python app

branliu0 commented 10 years ago

Just double checking, do you see a line at the beginning of the compilation that says something like 'Downloading blas/atlas binaries'?

This line should be setting the environment variables for BLAS: https://github.com/thenovices/heroku-buildpack-scipy/blob/master/bin/steps/numpy-scipy#L39

branliu0 commented 10 years ago

I also feel like there shouldn't be a need to recompile on the Heroku server, since the tarball already includes all the compiled files. I'm not sure why pip isn't picking that up though. Will look into it sometime...

ChristopherRabotin commented 10 years ago

I was able to correctly deploy after moving the scipy and numpy requirements to /requirements.txt. Until then, I was using pip syntax to point to another requirements file (-r reqs/prod.txt, as opposed to -r reqs/dev.txt).

shlomoa commented 9 years ago

http://stackoverflow.com/questions/25988796/cygwin-easy-install-scipy-error

branliu0 / heroku-buildpack-scipy

Is there any environment variable needed, apart from the BUILDPACK one? #1