search

freedomofpress / securethenews

An automated scanner and web dashboard for tracking TLS deployment across news organizations
https://securethe.news
GNU Affero General Public License v3.0
102 stars 25 forks source link

Installing nassl fails (pshtt dep) in vagrant using docker #96

Closed stedaniels closed 7 years ago

stedaniels commented 7 years ago

Up to date Ubuntu 16.04.1 bare server install. Installed vagrant, ansible, docker.io

TASK [securethenews-deploy : Install pshtt] **** fatal: [securethenews]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/pip install -e git+git://github.com/dhs-ncats/pshtt@7362a8bdabe7190a933e4c5e18561ba9d07dc561#egg=pshtt", "failed": true, "msg": "stdout: Obtaining pshtt from git+git://github.com/dhs-ncats/pshtt@7362a8bdabe7190a933e4c5e18561ba9d07dc561#egg=pshtt\n Cloning git://github.com/dhs-ncats/pshtt (to 7362a8bdabe7190a933e4c5e18561ba9d07dc561) to ./src/pshtt\nRequirement already satisfied: requests>=2.10.0 in /usr/local/lib/python3.4/dist-packages (from pshtt)\nCollecting sslyze>=0.13.6 (from pshtt)\n Downloading SSLyze-0.14.2.tar.gz (1.1MB)\nCollecting wget>=3.2 (from pshtt)\n Downloading wget-3.2.zip\nCollecting docopt (from pshtt)\n Downloading docopt-0.6.2.tar.gz\nCollecting requests_cache (from pshtt)\n Downloading requests_cache-0.4.13-py2.py3-none-any.whl\nCollecting nassl<0.15.0,>=0.14.0 (from sslyze>=0.13.6->pshtt)\n Downloading nassl-0.14.2.tar.gz (15.2MB)\n Complete output from command python setup.py egg_info:\n Traceback (most recent call last):\n File \"\", line 1, in \n File \"/tmp/pip-build-um741sgh/nassl/setup.py\", line 8, in \n from nassl import author, version\n File \"/tmp/pip-build-um741sgh/nassl/nassl/init.py\", line 24\n SSL_MODE_SEND_FALLBACK_SCSV = 0x00000080L\n ^\n SyntaxError: invalid syntax\n \n ----------------------------------------\n\n:stderr: Could not find a tag or branch '7362a8bdabe7190a933e4c5e18561ba9d07dc561', assuming commit.\nCommand \"python setup.py egg_info\" failed with error code 1 in /tmp/pip-build-um741sgh/nassl/\n"}

PLAY RECAP ***** securethenews : ok=17 changed=12 unreachable=0 failed=1

Ansible failed to complete successfully. Any error output should be visible above. Please fix these errors and try again. Makefile:8: recipe for target 'dev' failed make: *** [dev] Error 1

Full log:

steve@ubuntu-dev:~/securethenews$ make dev make ansible make[1]: Entering directory '/home/steve/securethenews' ansible-galaxy install -r ansible/requirements.yml -p ansible/roles

PLAY [Deploy Secure The News development machine] **

TASK [setup] *** ok: [securethenews]

TASK [geerlingguy.nodejs : include] **** skipping: [securethenews]

TASK [geerlingguy.nodejs : include] **** included: /home/steve/securethenews/ansible/roles/geerlingguy.nodejs/tasks/setup-Debian.yml for securethenews

TASK [geerlingguy.nodejs : Ensure apt-transport-https is installed.] *** changed: [securethenews]

TASK [geerlingguy.nodejs : Add Nodesource apt key.] **** changed: [securethenews]

TASK [geerlingguy.nodejs : Add NodeSource repositories for Node.js.] *** changed: [securethenews] => (item=deb https://deb.nodesource.com/node_4.x jessie main) changed: [securethenews] => (item=deb-src https://deb.nodesource.com/node_4.x jessie main)

TASK [geerlingguy.nodejs : Update apt cache if repo was added.] **** ok: [securethenews]

TASK [geerlingguy.nodejs : Ensure Node.js and npm are installed.] ** changed: [securethenews]

TASK [geerlingguy.nodejs : Define nodejs_install_npm_user] ***** ok: [securethenews]

TASK [geerlingguy.nodejs : Create npm global directory] **** changed: [securethenews]

TASK [geerlingguy.nodejs : Add npm_config_prefix bin directory to global $PATH.] *** changed: [securethenews]

TASK [geerlingguy.nodejs : Ensure npm global packages are installed.] **

TASK [geerlingguy.nodejs : Ensure npm global packages are at the latest release.] ***

TASK [securethenews-deploy : Install apt dependencies.] **** changed: [securethenews] => (item=[u'aptitude', u'git', u'ipython3', u'libjpeg-dev', u'postgresql-server-dev-all', u'python-dev', u'python-pip', u'python3-pip', u'ruby', u'tmux', u'vim', u'zlib1g-dev'])

TASK [securethenews-deploy : Upgrade all packages.] **** ok: [securethenews]

TASK [securethenews-deploy : Upgrade pip for Python 2 to avoid ImportError] **** changed: [securethenews]

TASK [securethenews-deploy : Upgrade pip for Python 3 to avoid ImportError] **** changed: [securethenews]

TASK [securethenews-deploy : Install Django project requirements] ** changed: [securethenews]

TASK [securethenews-deploy : Install Node dependencies] **** changed: [securethenews]

TASK [securethenews-deploy : Install gulp globally] **** changed: [securethenews]

TASK [securethenews-deploy : Install pshtt] **** fatal: [securethenews]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/pip install -e git+git://github.com/dhs-ncats/pshtt@7362a8bdabe7190a933e4c5e18561ba9d07dc561#egg=pshtt", "failed": true, "msg": "stdout: Obtaining pshtt from git+git://github.com/dhs-ncats/pshtt@7362a8bdabe7190a933e4c5e18561ba9d07dc561#egg=pshtt\n Cloning git://github.com/dhs-ncats/pshtt (to 7362a8bdabe7190a933e4c5e18561ba9d07dc561) to ./src/pshtt\nRequirement already satisfied: requests>=2.10.0 in /usr/local/lib/python3.4/dist-packages (from pshtt)\nCollecting sslyze>=0.13.6 (from pshtt)\n Downloading SSLyze-0.14.2.tar.gz (1.1MB)\nCollecting wget>=3.2 (from pshtt)\n Downloading wget-3.2.zip\nCollecting docopt (from pshtt)\n Downloading docopt-0.6.2.tar.gz\nCollecting requests_cache (from pshtt)\n Downloading requests_cache-0.4.13-py2.py3-none-any.whl\nCollecting nassl<0.15.0,>=0.14.0 (from sslyze>=0.13.6->pshtt)\n Downloading nassl-0.14.2.tar.gz (15.2MB)\n Complete output from command python setup.py egg_info:\n Traceback (most recent call last):\n File \"\", line 1, in \n File \"/tmp/pip-build-um741sgh/nassl/setup.py\", line 8, in \n from nassl import author, version\n File \"/tmp/pip-build-um741sgh/nassl/nassl/init.py\", line 24\n SSL_MODE_SEND_FALLBACK_SCSV = 0x00000080L\n ^\n SyntaxError: invalid syntax\n \n ----------------------------------------\n\n:stderr: Could not find a tag or branch '7362a8bdabe7190a933e4c5e18561ba9d07dc561', assuming commit.\nCommand \"python setup.py egg_info\" failed with error code 1 in /tmp/pip-build-um741sgh/nassl/\n"}

PLAY RECAP ***** securethenews : ok=17 changed=12 unreachable=0 failed=1

Ansible failed to complete successfully. Any error output should be visible above. Please fix these errors and try again. Makefile:8: recipe for target 'dev' failed make: *** [dev] Error 1

garrettr commented 7 years ago

Hm, looks like you're missing openssl-dev headers? Although if that's the cause, I'm surprised we haven't run in to this before.

garrettr commented 7 years ago

FWIW, we've only tested the Docker provider on Qubes (using Fedora 24).

garrettr commented 7 years ago

Based on my recent experiences installing stuff that relies on Pip wheels (which nassl appears to recommend as the primary way to install: https://github.com/nabla-c0d3/nassl), another possible fix is to update pip to the latest version before installing the Python dependencies.

stedaniels commented 7 years ago

Tried libssl-dev and libcurl4-openssl-dev in ansible/roles/securethenews-deploy/defaults/main.yml neither help. I'll try and install nassl in the container manually. Thanks for being responsive

garrettr commented 7 years ago

The error here is:

File "/tmp/pip-build-um741sgh/nassl/setup.py", line 8, in \n from nassl import author, version\n File "/tmp/pip-build-um741sgh/nassl/nassl/init.py", line 24\n SSL_MODE_SEND_FALLBACK_SCSV = 0x00000080L\n ^\n SyntaxError: invalid syntax\n \n

Based on looking into porting nassl to Python 3, this looks like a Python 2/3 problem. Long literals are cool in Python 2, but no longer exist in Python 3 because all integers are long.

stedaniels commented 7 years ago

Yeah, it installs fine with Python 2.7. sudo pip2.7 install nassl :-) But pshtt must want the 3 version?

garrettr commented 7 years ago

@stedaniels No, pshtt is actually Python 2.x (due to the nassl dependency). It's a bit of pain because our Django web application is Python 3 (because, well, it's 2017). Handling a dual Python 2/3 environment has been a consistent source of pain, and this appears to be more of the same.

Oh, is it because Python 2 isn't installed by default on xenial?!?

garrettr commented 7 years ago

Oh, is it because Python 2 isn't installed by default on xenial?!?

nvm, I was wrong about this. Too many Linux distributions to keep track of...

stedaniels commented 7 years ago

It's failing in the docker container which is Debian Jessie. It's after midnight here so I'm going to bed and I'll use a fresh pair of eyes over the weekend. The rest of the code/environment you've provided looks excellent! Great work :-)

garrettr commented 7 years ago

Thanks! Since you're on Ubuntu, you may have better luck with either of the virtualbox or libvirt providers.

Unassigning myself since I don't have an environment where I can reproduce this. I will reassign if/when I find the time to try to reproduce. Alternatively, @stedaniels if you figured this out you would be my hero!