search

tintinweb / scapy-ssl_tls

SSL/TLS layers for scapy the interactive packet manipulation tool
GNU General Public License v2.0
419 stars 156 forks source link

Broken setup #98

Closed CristianVladescu closed 7 years ago

CristianVladescu commented 7 years ago

Not sure how setuptools works, but it looks like the latest pip does not copy the data files to scapy. Instead it creates a new package with the name scapy_ssl_tls.

This is the result:

root@lithium:~# ls -la /opt/Python-2.7.13/lib/python2.7/site-packages/
total 1500
drwxr-xr-x 37 root root   4096 Jun 14 23:09 .
drwxr-xr-x 28 root root  20480 Jun 14 23:09 ..
drwxr-xr-x  3 root root   4096 Jun 14 23:09 asn1crypto
drwxr-xr-x  2 root root   4096 Jun 14 23:09 asn1crypto-0.22.0.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 cffi
drwxr-xr-x  2 root root   4096 Jun 14 23:09 cffi-1.10.0.dist-info
-rwxr-xr-x  1 root root 544904 Jun 14 23:09 _cffi_backend.so
drwxr-xr-x 10 root root   4096 Jun 14 23:09 Crypto
drwxr-xr-x  4 root root   4096 Jun 14 23:09 cryptography
drwxr-xr-x  2 root root   4096 Jun 14 23:09 cryptography-1.9.dist-info
-rw-r--r--  1 root root    126 Jun 14 23:09 easy_install.py
-rw-r--r--  1 root root    315 Jun 14 23:09 easy_install.pyc
drwxr-xr-x  2 root root   4096 Jun 14 23:09 enum
drwxr-xr-x  2 root root   4096 Jun 14 23:09 enum34-1.1.6.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 idna
drwxr-xr-x  2 root root   4096 Jun 14 23:09 idna-2.5.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 ipaddress-1.0.18.dist-info
-rw-r--r--  1 root root  80156 Jun 14 23:09 ipaddress.py
-rw-r--r--  1 root root  75662 Jun 14 23:09 ipaddress.pyc
drwxr-xr-x  2 root root   4096 Jun 14 23:09 .libs_cffi_backend
drwxr-xr-x  2 root root   4096 Jun 14 23:09 OpenSSL
drwxr-xr-x  3 root root   4096 Jun 14 23:09 opt
drwxr-xr-x 10 root root   4096 Jun 14 23:09 pip
drwxr-xr-x  2 root root   4096 Jun 14 23:09 pip-9.0.1.dist-info
drwxr-xr-x  4 root root   4096 Jun 14 23:09 pkg_resources
drwxr-xr-x  3 root root   4096 Jun 14 23:09 pycparser
drwxr-xr-x  2 root root   4096 Jun 14 23:09 pycparser-2.17.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 pycrypto-2.6.1.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 pyOpenSSL-17.0.0.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 readline-6.2.4.1.dist-info
-rwxr-xr-x  1 root root 578573 Jun 14 23:09 readline.so
-rw-r--r--  1 root root    119 Jun 14 23:08 README
drwxr-xr-x  8 root root   4096 Jun 14 23:09 scapy
drwxr-xr-x  2 root root   4096 Jun 14 23:09 scapy-2.3.2.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 scapy-2.3.3.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 scapy_ssl_tls
drwxr-xr-x  2 root root   4096 Jun 14 23:09 scapy_ssl_tls-1.2.3.2.dist-info
drwxr-xr-x  4 root root   4096 Jun 14 23:09 setuptools
drwxr-xr-x  2 root root   4096 Jun 14 23:09 setuptools-36.0.1.dist-info
drwxr-xr-x  2 root root   4096 Jun 14 23:09 six-1.10.0.dist-info
-rw-r--r--  1 root root  30098 Jun 14 23:09 six.py
-rw-r--r--  1 root root  29563 Jun 14 23:09 six.pyc
drwxr-xr-x  2 root root   4096 Jun 14 23:09 tinyec
drwxr-xr-x  2 root root   4096 Jun 14 23:09 tinyec-0.3.1.dist-info
drwxr-xr-x  5 root root   4096 Jun 14 23:09 wheel
drwxr-xr-x  2 root root   4096 Jun 14 23:09 wheel-0.29.0.dist-info

after executing this custom location Python install:

    echo "Install Python 2.7.13 ..."
    # Dependencies: build-essential libssl-dev python-setuptools python2.7-dev zlib1g-dev libncurses5-dev libffi-dev
    rm -r /opt/Python-2.7.13
    wget --no-check-certificate https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tgz -O /tmp/Python-2.7.13.tgz
    tar -zxf /tmp/Python-2.7.13.tgz -C /tmp
    pushd /tmp/Python-2.7.13
    ./configure --prefix=/opt/Python-2.7.13 --enable-unicode=ucs4 > /dev/null && make > /dev/null && make altinstall > /dev/null
    popd
    rm -r /tmp/Python-2.7.13*

    ln -s /opt/Python-2.7.13/bin/python2.7 /opt/Python-2.7.13/bin/python
    initialPath=$PATH
    initialPythonUserBase=$PYTHONUSERBASE
    export PATH=/opt/Python-2.7.13/bin:$PATH
    export PYTHONUSERBASE=/opt/Python-2.7.13
    wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py -O - | python - --user
    export PYTHONUSERBASE=$initialPythonUserBase
    pip install readline -t /opt/Python-2.7.13/lib/python2.7/site-packages/
    pip install pyOpenSSL -t /opt/Python-2.7.13/lib/python2.7/site-packages/
    pip install scapy -t /opt/Python-2.7.13/lib/python2.7/site-packages/
    pip install scapy-ssl_tls -t /opt/Python-2.7.13/lib/python2.7/site-packages/ ;# setup.py for this package does not work well so we'll have to manually move files
    rm /opt/Python-2.7.13/lib/python2.7/site-packages/scapy_ssl_tls/__init__*
    mv /opt/Python-2.7.13/lib/python2.7/site-packages/scapy_ssl_tls/* /opt/Python-2.7.13/lib/python2.7/site-packages/scapy/layers/
    rm -r /opt/Python-2.7.13/lib/python2.7/site-packages/scapy_ssl_tls*
    export PATH=$initialPath

And this is what data_files=get_layer_files_dst(get_site_packages()) returns:

[('/opt/Python-2.7.13/lib/python2.7/site-packages/scapy/layers', ['scapy_ssl_tls/pkcs7.py', 'scapy_ssl_tls/ssl_tls_crypto.py', 'scapy_ssl_tls/ssl_tls.py', 'scapy_ssl_tls/ssl_tls_registry.py', 'scapy_ssl_tls/ssl_tls_automata.py'])]
tintinweb commented 7 years ago

Hi @CristianVladescu, appreciate your report!

When installing scapy-ssl_tls with pip or setup.py setuptools is creating a python package in the site-packages directory. To make layer files automatically available to scapy we're also copying relevant layer files to scapy/layers and patching the layer loader to include ssl_tls.

In your case you are building a version of python to /opt/Python-2.7.13 and then installing scapy-ssl_tls to that python environment. Installing it as a python package is by intention and makes the package available to other python projects. However, in your case you seem to mix up python versions/environments and I am not quite sure if it was intendet the way you're using it. I would suggest to run /opt/Python-2.7.13/bin/python2.7 -m pip install setuptools (make sure to have setuptools installed and compile python with ssl support, you'll probably need it anyway for all the crypto) to force pip to run with the custom python version and install the packages from there.

Installing from one python environment to the other using pip -t (this is running system's python interpreter): The main problem we have with pip -t is that we do not seem to have the custom target destination available in setup.py. Maybe this will get fixed when moving to python whl instead. Right now we try to find scapys location in the current python interpreters user and system site-packages location.

Please try installing it by running pip with the python environment from /opt and report back if that resolves your problems. Virtualenv might also be an alternative for you (see ci-runtests.sh which installs scapy-ssl_tls to a venv)

tin

tintinweb commented 7 years ago

feel free to reopen this issue if you feel like there's something we can do on our side.