search

pydata / bottleneck

Fast NumPy array functions written in C
BSD 2-Clause "Simplified" License
1.05k stars 101 forks source link

ERROR: Could not build wheels for bottleneck which use PEP 517 and cannot be installed directly #348

Open AlysonR opened 4 years ago

AlysonR commented 4 years ago

I get the errors ERROR: Failed building wheel ERROR: Could not build wheels for bottleneck which use PEP 517 and cannot be installed directly

I have python-devel installed, my wheel is updated to the current version, pip is updated to the current version, pep517 is installed

I saw this was an issue before that was closed after a false answer was given that was never proven to work (installing developers tools). I will not use conda or anaconda so please do not suggest that I install those to solve this problem.

qwhelan commented 4 years ago

Could you please provide exact versions of each of those dependencies, as well as the operating system?

ArasAzimi commented 4 years ago

@qwhelan I had this same issue:

Here are some dependencies and the virtual env I see the error in:

-Ubuntu 18.04.4 LTS -5.3.0-62-generic -x86_64 -Python 3.7.8 -pip 20.1.1

virtualenv -p python3.7 venv
source venv/bin/activate
pip install --upgrade setuptools wheel
pip install numpy==1.18.1
pip install fastai

This seems to be working with python 3.7.3 ( I had a mac system working and when I checked the python version it had 3.7.3). On the same Ubuntu system as above when I downgraded to python 3.7.3 The error no longer showed up. I thought I will post here in case it can help to find out the root cause. Also tried with python 3.8.3 and had the error.

qwhelan commented 4 years ago

@ArasAzimi Thank you, I will attempt to reproduce locally.

Zeitsperre commented 4 years ago

You might not find this useful, but we've noted the same failures on Python3.6 builds (on Travis CI) of the master branch of Bottleneck for the project xclim. I've opened a PR on there to try and figure out the cause: https://github.com/Ouranosinc/xclim/pull/517

Feel free to check out the build statuses on that PR; they seem to be exactly as @AlysonR describes.

aulemahal commented 3 years ago

I believe I have found the source of the error, at least in my configuration. It is due to the specific of virtual environment creation with python -m venv.

When creating a virtual environment with venv into ${venv}, a ${venv}/include folder is created and sys.prefix == ${venv}. However, in opposition to virtualenv, the include folder is empty, it exists solely for files created by packages installed within the environment. Source : PEP 405.

The include folders needed by bottleneck are still accessible, but their path has to be constructed relative to sys.base_prefix instead, to get the system files. Changing: https://github.com/pydata/bottleneck/blob/f2bc792e540908d165143c4930d63d1e7ab806d3/bottleneck/src/bn_config.py#L30 to

    for prefix in [sys.prefix, sys.exec_prefix, sys.base_prefix]:

fixes the problem.

Of course, I have pythonX.X-dev installed (on ubuntu) for this to work.

EDIT: I have python 3.7.9.

AndyWangSFU commented 3 years ago

This is really a weird issue. It happens in my local machine, and happens again in my poetry virtual machine... (BTW they have different python versions)

nocluebutalotofit commented 3 years ago

On macOS Big Sur 11.2.3 I get the following error running pip install:

$ pip install Bottleneck==1.3.2

Collecting Bottleneck==1.3.2 Using cached Bottleneck-1.3.2.tar.gz (88 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Collecting numpy Using cached numpy-1.20.2-cp38-cp38-macosx_10_9_x86_64.whl (16.0 MB) Building wheels for collected packages: Bottleneck Building wheel for Bottleneck (PEP 517) ... error ERROR: Command errored out with exit status 1: command: /Users/daniel/PycharmProjects/machine-learning-for-trading_book/venv/bin/python /Users/daniel/PycharmProjects/machine-learning-for-trading_book/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/hh/ll8cmcb100v7lbrfh1n1t96m0000gn/T/tmpd5gppf3b cwd: /private/var/folders/hh/ll8cmcb100v7lbrfh1n1t96m0000gn/T/pip-install-082vgn/bottleneck_c8de5d9f837747fba5e8843c02aa359c Complete output (127 lines): running bdist_wheel running build running build_py creating build creating build/lib.macosx-10.14.6-x86_64-3.8 creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck copying bottleneck/_version.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck copying bottleneck/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck copying bottleneck/_pytesttester.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark copying bottleneck/benchmark/bench_detailed.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark copying bottleneck/benchmark/autotimeit.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark copying bottleneck/benchmark/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark copying bottleneck/benchmark/bench.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/benchmark creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow copying bottleneck/slow/reduce.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow copying bottleneck/slow/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow copying bottleneck/slow/nonreduce.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow copying bottleneck/slow/move.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow copying bottleneck/slow/nonreduce_axis.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/slow creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/nonreduce_axis_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/scalar_input_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/reduce_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/util.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/move_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/input_modification_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/nonreduce_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/list_input_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests copying bottleneck/tests/memory_test.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/tests creating build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src copying bottleneck/src/bn_config.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src copying bottleneck/src/init.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src copying bottleneck/src/bn_template.py -> build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/src UPDATING build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/_version.py set build/lib.macosx-10.14.6-x86_64-3.8/bottleneck/_version.py to '1.3.2' running build_ext running config compiling '_configtest.c':

pragma clang diagnostic error "-Wattributes"

int attribute((optimize("O3"))) have_attribute_optimize_opt_3(void*);

int main(void) { return 0; }

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o clang-11: error: the clang compiler does not support '-march=core2' failure. removing: _configtest.c _configtest.o compiling '_configtest.c':

ifndef __cplusplus

static inline int static_func (void) { return 0; } inline int nostatic_func (void) { return 0; }

endif

int main(void) { int r1 = static_func(); int r2 = nostatic_func(); return r1 + r2; }

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o clang-11: error: the clang compiler does not support '-march=core2' failure. removing: _configtest.c _configtest.o compiling '_configtest.c':

ifndef __cplusplus

static inline int static_func (void) { return 0; } inline int nostatic_func (void) { return 0; }

endif

int main(void) { int r1 = static_func(); int r2 = nostatic_func(); return r1 + r2; }

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o clang-11: error: the clang compiler does not support '-march=core2' failure. removing: _configtest.c _configtest.o compiling '_configtest.c':

ifndef __cplusplus

static __inline int static_func (void) { return 0; } __inline int nostatic_func (void) { return 0; }

endif

int main(void) { int r1 = static_func(); int r2 = nostatic_func(); return r1 + r2; }

x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -c _configtest.c -o _configtest.o clang-11: error: the clang compiler does not support '-march=core2' failure. removing: _configtest.c _configtest.o building 'bottleneck.reduce' extension creating build/temp.macosx-10.14.6-x86_64-3.8 creating build/temp.macosx-10.14.6-x86_64-3.8/bottleneck creating build/temp.macosx-10.14.6-x86_64-3.8/bottleneck/src x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/daniel/opt/anaconda3/include -D_FORTIFY_SOURCE=2 -isystem /Users/daniel/opt/anaconda3/include -I/private/var/folders/hh/ll8cmcb100v7lbrfh1n1t96m0000gn/T/pip-build-env-6idcz14p/overlay/lib/python3.8/site-packages/numpy/core/include -I/Users/daniel/PycharmProjects/machine-learning-for-trading_book/venv/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -Ibottleneck/src -c bottleneck/src/reduce.c -o build/temp.macosx-10.14.6-x86_64-3.8/bottleneck/src/reduce.o -O2 clang-11: error: the clang compiler does not support '-march=core2' error: command 'x86_64-apple-darwin13.4.0-clang' failed with exit status 1

ERROR: Failed building wheel for Bottleneck Failed to build Bottleneck ERROR: Could not build wheels for Bottleneck which use PEP 517 and cannot be installed directly

bkaankuguoglu commented 3 years ago

Is there any update regarding this issue? So far I've tried the following but none worked:

brentrichards commented 3 years ago

After many re-installs, running: pip install --upgrade setuptools wheel fixed the issue for me. (On Windows Dev edition).

profxj commented 3 years ago

Running on a VM via kubernetes with conda installed using

# Miniconda
ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.9.2-Linux-x86_64.sh \
    && mkdir /root/.conda \
    && bash Miniconda3-py38_4.9.2-Linux-x86_64.sh -b \
    && rm -f Miniconda3-py38_4.9.2-Linux-x86_64.sh

fails to Install, even after the

pip install --upgrade setuptools wheel

approach suggested above.

The final bit of the crash:

  gcc -pthread -B /root/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
  failure.
  removing: _configtest.c _configtest.o
  building 'bottleneck.reduce' extension
  creating build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/bottleneck
  creating build/temp.linux-x86_64-3.8/bottleneck/src
  gcc -pthread -B /root/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/pip-build-env-xf8ufge7/overlay/lib/python3.8/site-packages/numpy/core/include -I/root/miniconda3/include/python3.8 -Ibottleneck/src -c bottleneck/src/reduce.c -o build/temp.linux-x86_64-3.8/bottleneck/src/reduce.o -O2
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for bottleneck
Failed to build bottleneck
ERROR: Could not build wheels for bottleneck which use PEP 517 and cannot be installed directly
brentrichards commented 3 years ago

On Windows, I finally realised that I needed a background tool to build the Wheel, that is not installed by default (and the message doen't hint at). I installed Visual Studio Build tools, and it all worked.

franciscorodriguez92 commented 3 years ago

Hi!

Installing gcc solved the problem for me (Ubuntu 20.04):

sudo apt-get install gcc

jweisbaum commented 3 years ago

Has anyone been able to get this to work on mac? Trying to install with Python 3.9 in the Blender installation.

johncox00 commented 2 years ago

I came across this issue while searching for a similar issue installing implicit wherein I got...

ERROR: Could not build wheels for implicit which use PEP 517 and cannot be installed directly

@jweisbaum I was able to get past the problem on my mac with brew install gcc and then pip install ...