scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.
https://scrapy.org
BSD 3-Clause "New" or "Revised" License
51.16k stars 10.35k forks source link

' error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ' #2115

Closed euler16 closed 7 years ago

euler16 commented 7 years ago

I wanted to install scrapy in virtualenv using pip (Python 3.5) but I get the following error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

I tried with Python 2.7 but I get the same error scrapy

eliasdorneles commented 7 years ago

Have you tried installing the libraries mentioned in the documentation? Here is the link for Ubuntu: http://doc.scrapy.org/en/latest/intro/install.html#ubuntu-9-10-or-above

redapple commented 7 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip
redapple commented 7 years ago

This should get you covered (tested with Ubuntu 16.04):

sudo apt-get update -y

# install python3 and dev headers
sudo apt-get install -y python3 python3-dev

# pip is great
sudo apt-get install -y python-pip

# for cryptography (required by twisted[tls])
sudo apt-get install -y libssl-dev libffi-dev

# for lxml
sudo apt-get install -y libxml2-dev libxslt1-dev

sudo pip install virtualenvwrapper
source /usr/local/bin/virtualenvwrapper.sh

# create a Python3 virtualenv
mkvirtualenv -p python3 scrapy.py3

# and install scrapy inside it
(scrapy.py3) ~$ pip install scrapy

Check that you're using Python3:

(scrapy.py3) ~$  scrapy version -v
Scrapy    : 1.1.0
lxml      : 3.6.0.0
libxml2   : 2.9.3
Twisted   : 16.3.0
Python    : 3.5.1+ (default, Mar 30 2016, 22:46:26) - [GCC 5.3.1 20160330]
pyOpenSSL : 16.0.0 (OpenSSL 1.0.2g-fips  1 Mar 2016)
Platform  : Linux-4.4.0-28-generic-x86_64-with-Ubuntu-16.04-xenial
redapple commented 7 years ago

@euler16 , did you manage to get it working?

iamrudraksha commented 7 years ago

Well it worked for me. Thanks! :smile:

redapple commented 7 years ago

Great to hear! Closing.

ko4ergins commented 6 years ago

Linux Mint X64 working too. Thank you!!!

Scrapy : 1.4.0 lxml : 4.1.0.0 libxml2 : 2.9.5 cssselect : 1.0.1 parsel : 1.2.0 w3lib : 1.18.0 Twisted : 17.9.0 Python : 3.5.2 (default, Sep 14 2017, 22:51:06) - [GCC 5.4.0 20160609] pyOpenSSL : 17.3.0 (OpenSSL 1.1.0f 25 May 2017) Platform : Linux-4.10.0-37-generic-x86_64-with-LinuxMint-18.2-sonya

anujgupta82 commented 6 years ago

On Ubuntu 16.04 I had to do

sudo apt-get install libblas-dev libatlas-base-dev
colllin commented 5 years ago

All I needed was python-dev.

$ sudo apt install python3.6-dev
ranoloel commented 5 years ago

Hello, I have also a problem in installing pylibmc in python3.4. Had the same problem output. image

deanmcginndm commented 4 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

I love you very much because you gave a great answer!!!!

songhuili commented 4 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

hello,when i pip install matplotlib on jestson nano ,it gave me the same problem.But it did't work for me... can you give me some help?thanks!

samhithaaaa commented 4 years ago

Great to hear! Closing.

Hi,I am facing failed building wheel for grpcio errorLinkError: command 'aarch64-linux-gnu-g++' failed with exit status 1ERROR: Command errored out with exit status 1: /home/tf/venv/venc-tf-1.14/bin/python2 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-WT9TE4/grpcio/setup.py'"'"'; file='"'"'/tmp/pip-install-WT9TE4/grpcio/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-otPtXv/install-record.txt --single-version-externally-managed --compile --install-headers /home/tf/venv/venc-tf-1.14/include/site/python2.7/grpcio Check the logs for full command output. could you please help me?

div338 commented 4 years ago

All I needed was python-dev.

$ sudo apt install python3.6-dev

This reply by colllin worked the best for me.

damodharn commented 4 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

it worked for me. not for scrapy but for same type of error with psycopg2 installation.

icaropablo2 commented 4 years ago

Awesome, I had this issue after update to Ubuntu 19. So I just used

sudo apt-get install python2.7-dev

Thank you for sharing.

lobstrio commented 4 years ago

All I needed was python-dev.

$ sudo apt install python3.6-dev
sudo apt install python3.7-dev 

if you recently upgraded python

Abeautifulsnow commented 4 years ago

@euler16 , for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

hello,when i pip install matplotlib on jestson nano ,it gave me the same problem.But it did't work for me... can you give me some help?thanks!

Wow, great. I have solved my problems by your answer. This problem has plagued me for a long time.

LittleFlyFish commented 4 years ago

All I needed was python-dev.

$ sudo apt install python3.6-dev
sudo apt install python3.7-dev 

if you recently upgraded python

That‘s solved my problem !!! Thank you so much!

danbaghilovici commented 4 years ago

All I needed was python-dev.

$ sudo apt install python3.6-dev
sudo apt install python3.7-dev 

if you recently upgraded python

the same goes for python 3.8: sudo apt install python3.8-dev

davide1995 commented 4 years ago

What about if I don't have the permissions for sudo and I'm installing the package in the .local folder with pip3 install --user ?

Gallaecio commented 4 years ago

If you need the python headers installed (what python3.8-dev provides), and cannot install them system-wide, see https://superuser.com/q/431347/172753

mohammad-aqajani commented 4 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

I have this problem with installing "pyemd" module, it resolved! thanks

smehdia commented 3 years ago

I had the same problem, updating pip3 solved my problem: sudo -H pip3 install --upgrade pip

Chealkeo commented 3 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

thanks for your solution!

naXa777 commented 3 years ago

python-pip

Ubuntu 20.04 + Python3

apt can't locate python-pip package. otherwise, without this line, your solution works great!

alok87 commented 3 years ago

For people facing similar issues in Ubuntu14.04 Locking to this version works cryptography==2.6.1

themagicalmammal commented 3 years ago

How to fix

Collecting pyaudio Using cached https://files.pythonhosted.org/packages/ab/42/b4f04721c5c5bfc196ce156b3c768998ef8c0ae3654ed29ea5020c749a6b/PyAudio-0.2.11.tar.gz Building wheels for collected packages: pyaudio Running setup.py bdist_wheel for pyaudio ... error Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-qzr8qlsr/pyaudio/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/tmp_m4yhwa3pip-wheel- --python-tag cp36: running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-3.6 copying src/pyaudio.py -> build/lib.linux-x86_64-3.6 running build_ext building '_portaudio' extension creating build/temp.linux-x86_64-3.6 creating build/temp.linux-x86_64-3.6/src x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c src/_portaudiomodule.c -o build/temp.linux-x86_64-3.6/src/_portaudiomodule.o src/_portaudiomodule.c:29:10: fatal error: portaudio.h: No such file or directory

include "portaudio.h"

        ^~~~~~~~~~~~~

compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1


Failed building wheel for pyaudio Running setup.py clean for pyaudio Failed to build pyaudio Installing collected packages: pyaudio Running setup.py install for pyaudio ... error Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-qzr8qlsr/pyaudio/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-51povmlz-record/install-record.txt --single-version-externally-managed --compile --user --prefix=: running install running build running build_py creating build creating build/lib.linux-x86_64-3.6 copying src/pyaudio.py -> build/lib.linux-x86_64-3.6 running build_ext building '_portaudio' extension creating build/temp.linux-x86_64-3.6 creating build/temp.linux-x86_64-3.6/src x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c src/_portaudiomodule.c -o build/temp.linux-x86_64-3.6/src/_portaudiomodule.o src/_portaudiomodule.c:29:10: fatal error: portaudio.h: No such file or directory

include "portaudio.h"

          ^~~~~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-qzr8qlsr/pyaudio/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-51povmlz-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-qzr8qlsr/pyaudio/

sl-alexandreaudibert commented 3 years ago

To me, i needed this : sudo apt install unixodbc-dev

themagicalmammal commented 3 years ago

I had to use the .whl file.

Garib-ka-code commented 3 years ago

I had the same problem, updating pip3 solved my problem: sudo -H pip3 install --upgrade pip

Solved my problem

flaviaerius commented 3 years ago

None of the solutions from all of you solved this problem for me (same, but for pandas and cyvcf2). I am trying to build a docker image with python, and when I install the requirements, during wheel building, I get the error:

error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
wanting4 commented 3 years ago

I had the same problem, updating pip3 solved my problem: sudo -H pip3 install --upgrade pip

Only this works for me. Thank you!

hellodit commented 3 years ago

None of the solutions from all of you solved this problem for me (same, but for pandas and cyvcf2). I am trying to build a docker image with python, and when I install the requirements, during wheel building, I get the error:

error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

Me as well, using WSL ubuntu 20

HamishLacmane commented 3 years ago

Hi All, the error with x86_64-linux-gnu-gcc suggest gcc (c compiler I think) is not installed. sudo apt-get install gcc

The above fixed it for me!

Have a good day!

valueworker commented 3 years ago

Hi, an error occurred when installing the charm-crypto==0.43 package on Ubuntu 20.04 and using the make operation error: command'x86_64-linux-gnu-gcc' failed with exit status 1. None of the above methods worked. Who can tell me how to solve it, thanks

twitt commented 3 years ago

Hi All, the error with x86_64-linux-gnu-gcc suggest gcc (c compiler I think) is not installed. sudo apt-get install gcc

The above fixed it for me!

Have a good day!

the only solution that worked for me

valueworker commented 3 years ago

大家好,未安装建议gcc(我认为是c编译器) 的错误x86_64-linux-gnu-gccsudo apt-get install gcc 以上为我修复了它! 祝你有美好的一天!

唯一对我有用的解决方案

大家好,未安装建议gcc(我认为是c编译器) 的错误x86_64-linux-gnu-gccsudo apt-get install gcc 以上为我修复了它! 祝你有美好的一天!

唯一对我有用的解决方案

The latest version of gcc has been installed, but still the same error

erfanmosaddeghi commented 3 years ago

@euler16 ,

for scrapy with Python 3, you'll need

sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

in python3:

sudo apt-get install python3 python3-dev \ build-essential libssl-dev libffi-dev \ libxml2-dev libxslt1-dev zlib1g-dev \ python3-pip

andereb081 commented 3 years ago

Thanks for this:

@euler16 ,

for scrapy with Python 3, you'll need


sudo apt-get install python3 python-dev python3-dev \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

with Python 2, you'll need

sudo apt-get install python-dev  \
     build-essential libssl-dev libffi-dev \
     libxml2-dev libxslt1-dev zlib1g-dev \
     python-pip

For python3.8 I had to change the above to:

sudo apt-get install python3.8 python-dev python3.8-dev \
        build-essential libssl-dev libffi-dev \
        libxml2-dev libxslt1-dev zlib1g-dev \
        python-pip

Then the build command worked, I was working with thumbor on ubuntu-18-lts which kept throwing the same error as above

MoTechStore commented 2 years ago

I had the same problem, updating pip3 solved my problem: sudo -H pip3 install --upgrade pip

This works for me too