search

DanMcInerney / xsscrapy

XSS spider - 66/66 wavsep XSS detected
1.66k stars 441 forks source link

Difficulty installing dependencies #20

Closed sdsunjay closed 9 years ago

sdsunjay commented 9 years ago

CentOS 6.5 32-bit

pip.log

Requirement already satisfied (use --upgrade to upgrade): Scrapy==0.24.4 in /usr/lib/python2.6/site-packages (from -r requirements.txt (line 1))

Requirement already satisfied (use --upgrade to upgrade): pybloom==1.1 in /usr/lib/python2.6/site-packages (from -r requirements.txt (line 2))

Requirement already satisfied (use --upgrade to upgrade): requests in /usr/lib/python2.6/site-packages (from -r requirements.txt (line 3))

  skipping extra security
  skipping extra security
  skipping extra security
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup in /usr/lib/python2.6/site-packages (from -r requirements.txt (line 4))

Downloading/unpacking Twisted>=10.0.0 (from Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package Twisted

    running egg_info
    writing requirements to pip-egg-info/Twisted.egg-info/requires.txt
    writing pip-egg-info/Twisted.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/Twisted.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/Twisted.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/Twisted.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/Twisted.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/Twisted has version 15.0.0, which satisfies requirement Twisted>=10.0.0 (from Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking w3lib>=1.8.0 (from Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package w3lib

    /usr/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'zip_zafe'

      warnings.warn(msg)

    running egg_info
    writing requirements to pip-egg-info/w3lib.egg-info/requires.txt
    writing pip-egg-info/w3lib.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/w3lib.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/w3lib.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/w3lib.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/w3lib.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/w3lib has version 1.11.0, which satisfies requirement w3lib>=1.8.0 (from Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking queuelib (from Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package queuelib

    running egg_info
    writing pip-egg-info/queuelib.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/queuelib.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/queuelib.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/queuelib.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/queuelib.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/queuelib has version 1.2.2, which satisfies requirement queuelib (from Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking lxml (from Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package lxml

    /usr/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'bugtrack_url'

      warnings.warn(msg)

    Building lxml version 3.4.1.

    Building without Cython.

    Using build configuration of libxslt 1.1.26

    Building against libxml2/libxslt in the following directory: /usr/lib

    running egg_info
    writing requirements to pip-egg-info/lxml.egg-info/requires.txt
    writing pip-egg-info/lxml.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/lxml.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/lxml.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/lxml.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/lxml.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/lxml has version 3.4.1, which satisfies requirement lxml (from Scrapy==0.24.4->-r requirements.txt (line 1))
  skipping extra source
  skipping extra cssselect
  skipping extra html5
  skipping extra htmlsoup
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /root/.local/lib/python2.6/site-packages/pyOpenSSL-0.14-py2.6.egg (from Scrapy==0.24.4->-r requirements.txt (line 1))

Downloading/unpacking cssselect>=0.9 (from Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package cssselect

    running egg_info
    writing pip-egg-info/cssselect.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/cssselect.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/cssselect.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/cssselect.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'docs/_build'

    writing manifest file 'pip-egg-info/cssselect.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/cssselect has version 0.9.1, which satisfies requirement cssselect>=0.9 (from Scrapy==0.24.4->-r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /root/.local/lib/python2.6/site-packages/six-1.8.0-py2.6.egg (from Scrapy==0.24.4->-r requirements.txt (line 1))

Downloading/unpacking bitarray>=0.3.4 (from pybloom==1.1->-r requirements.txt (line 2))

  Running setup.py egg_info for package bitarray

    running egg_info
    writing pip-egg-info/bitarray.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/bitarray.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/bitarray.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/bitarray.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/bitarray.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/bitarray has version 0.8.1, which satisfies requirement bitarray>=0.3.4 (from pybloom==1.1->-r requirements.txt (line 2))
Downloading/unpacking zope.interface>=3.6.0 (from Twisted>=10.0.0->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package zope.interface

    running egg_info
    writing requirements to pip-egg-info/zope.interface.egg-info/requires.txt
    writing pip-egg-info/zope.interface.egg-info/PKG-INFO
    writing namespace_packages to pip-egg-info/zope.interface.egg-info/namespace_packages.txt
    writing top-level names to pip-egg-info/zope.interface.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/zope.interface.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/zope.interface.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no previously-included files matching '*.dll' found anywhere in distribution

    warning: no previously-included files matching '*.pyc' found anywhere in distribution

    warning: no previously-included files matching '*.pyo' found anywhere in distribution

    warning: no previously-included files matching '*.so' found anywhere in distribution

    writing manifest file 'pip-egg-info/zope.interface.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/zope.interface has version 4.1.2, which satisfies requirement zope.interface>=3.6.0 (from Twisted>=10.0.0->Scrapy==0.24.4->-r requirements.txt (line 1))
  skipping extra test
  skipping extra docs
  skipping extra docs
  skipping extra testing
  skipping extra testing
  skipping extra testing
Downloading/unpacking cryptography>=0.2.1 (from pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package cryptography

    running egg_info
    writing requirements to pip-egg-info/cryptography.egg-info/requires.txt
    writing pip-egg-info/cryptography.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt
    writing entry points to pip-egg-info/cryptography.egg-info/entry_points.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'docs/_build'

    warning: no previously-included files matching '*' found under directory 'vectors'

    writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/cryptography has version 0.7.2, which satisfies requirement cryptography>=0.2.1 (from pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): distribute in /usr/lib/python2.6/site-packages (from zope.interface>=3.6.0->Twisted>=10.0.0->Scrapy==0.24.4->-r requirements.txt (line 1))

Downloading/unpacking cffi>=0.8 (from cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package cffi

    compiling '_configtest.c':

    __thread int some_threadlocal_variable_42;

    running egg_info
    writing requirements to pip-egg-info/cffi.egg-info/requires.txt
    writing pip-egg-info/cffi.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/cffi.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/cffi.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/cffi.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/cffi.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/cffi has version 0.8.6, which satisfies requirement cffi>=0.8 (from cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking pyasn1 (from cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package pyasn1

    running egg_info
    writing pip-egg-info/pyasn1.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pyasn1.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pyasn1.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/pyasn1.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/pyasn1.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/pyasn1 has version 0.1.7, which satisfies requirement pyasn1 (from cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking enum34 (from cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package enum34

    running egg_info
    writing requirements to pip-egg-info/enum34.egg-info/requires.txt
    writing pip-egg-info/enum34.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/enum34.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/enum34.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/enum34.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/enum34.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/enum34 has version 1.0.4, which satisfies requirement enum34 (from cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking pycparser (from cffi>=0.8->cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package pycparser

    running egg_info
    writing pip-egg-info/pycparser.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pycparser.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pycparser.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/pycparser.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/pycparser.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/pycparser has version 2.10, which satisfies requirement pycparser (from cffi>=0.8->cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))
Downloading/unpacking ordereddict (from enum34->cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))

  Running setup.py egg_info for package ordereddict

    running egg_info
    writing pip-egg-info/ordereddict.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/ordereddict.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/ordereddict.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/ordereddict.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/ordereddict.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/ordereddict has version 1.1, which satisfies requirement ordereddict (from enum34->cryptography>=0.2.1->pyOpenSSL->Scrapy==0.24.4->-r requirements.txt (line 1))
Installing collected packages: Twisted, w3lib, queuelib, lxml, cssselect, bitarray, zope.interface, cryptography, cffi, pyasn1, enum34, pycparser, ordereddict

  Running setup.py install for Twisted

    Running command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/Twisted/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-6YgujV-record/install-record.txt --single-version-externally-managed
    running install
    running build
    running build_py
    running egg_info
    writing requirements to Twisted.egg-info/requires.txt
    writing Twisted.egg-info/PKG-INFO
    writing top-level names to Twisted.egg-info/top_level.txt
    writing dependency_links to Twisted.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'Twisted.egg-info/SOURCES.txt'
    writing manifest file 'Twisted.egg-info/SOURCES.txt'
    running build_ext
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c conftest.c -o conftest.o

    **building 'twisted.runner.portmap' extension**

    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c twisted/runner/portmap.c -o build/temp.linux-i686-2.6/twisted/runner/portmap.o

    **twisted/runner/portmap.c:10:20: error: Python.h: No such file or directory**

    twisted/runner/portmap.c:14: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token

    twisted/runner/portmap.c:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token

    twisted/runner/portmap.c:45: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PortmapMethods’

    twisted/runner/portmap.c: In function ‘initportmap’:

    twisted/runner/portmap.c:55: warning: implicit declaration of function ‘Py_InitModule’

    twisted/runner/portmap.c:55: error: ‘PortmapMethods’ undeclared (first use in this function)

    twisted/runner/portmap.c:55: error: (Each undeclared identifier is reported only once

    twisted/runner/portmap.c:55: error: for each function it appears in.)

    error: command 'gcc' failed with exit status 1

    Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/Twisted/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-6YgujV-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_py

running egg_info

writing requirements to Twisted.egg-info/requires.txt

writing Twisted.egg-info/PKG-INFO

writing top-level names to Twisted.egg-info/top_level.txt

writing dependency_links to Twisted.egg-info/dependency_links.txt

warning: manifest_maker: standard file '-c' not found

reading manifest file 'Twisted.egg-info/SOURCES.txt'

writing manifest file 'Twisted.egg-info/SOURCES.txt'

running build_ext

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c conftest.c -o conftest.o

building 'twisted.runner.portmap' extension

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c twisted/runner/portmap.c -o build/temp.linux-i686-2.6/twisted/runner/portmap.o

twisted/runner/portmap.c:10:20: error: Python.h: No such file or directory

twisted/runner/portmap.c:14: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token

twisted/runner/portmap.c:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token

twisted/runner/portmap.c:45: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PortmapMethods’

twisted/runner/portmap.c: In function ‘initportmap’:

twisted/runner/portmap.c:55: warning: implicit declaration of function ‘Py_InitModule’

twisted/runner/portmap.c:55: error: ‘PortmapMethods’ undeclared (first use in this function)

twisted/runner/portmap.c:55: error: (Each undeclared identifier is reported only once

twisted/runner/portmap.c:55: error: for each function it appears in.)

error: command 'gcc' failed with exit status 1

----------------------------------------

Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/Twisted/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-6YgujV-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/Twisted

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/usr/lib/python2.6/site-packages/pip/commands/install.py", line 271, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 1185, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 592, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python2.6/site-packages/pip/util.py", line 662, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/Twisted/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-6YgujV-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/Twisted
DanMcInerney commented 9 years ago

I think the problem is: error: command 'gcc' failed with exit status 1

My guess for a solution: yum install gcc