search

italia / spid-sp-test

SAML2 SPID/CIE Service Provider validation tool
European Union Public License 1.2
38 stars 17 forks source link

Problems installing and running spid_sp_test in virtual env #122

Closed mauromol closed 2 years ago

mauromol commented 2 years ago

I followed the instructions to create a python3 virtual env and install spid_sp_test. However, when I do pip install spid_sp_test after activating the virtual env, I get this:

ERROR: spid-compliant-certificates 0.4.1 has requirement cryptography==35.0.0, but you'll have cryptography 36.0.2 which is incompatible.

Then, I retried to install with pip install spid_sp_test --upgrade and then it seems like it gets installed:

[...]
Collecting cryptography==35.0.0
  Downloading cryptography-35.0.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.7 MB)
     |████████████████████████████████| 3.7 MB 4.8 MB/s
[...]
Installing collected packages: cryptography
  Attempting uninstall: cryptography
    Found existing installation: cryptography 36.0.2
    Uninstalling cryptography-36.0.2:
      Successfully uninstalled cryptography-36.0.2
Successfully installed cryptography-35.0.0

So, it seems like the second time it has installed the older version of the cryptography package without complaining, I don't know why it doesn't do this on the first attempt.

However, spid_sp_test does not start yet. I get the following exception:

(spid_sp_test) mauro@hppb ~/workspace/python-env $ spid_sp_test
Traceback (most recent call last):
  File "/home/mauro/workspace/python-env/spid_sp_test/bin/spid_sp_test", line 17, in <module>
    from spid_sp_test.response import SpidSpResponseCheck
  File "/home/mauro/workspace/python-env/spid_sp_test/lib/python3.8/site-packages/spid_sp_test/response.py", line 14, in <module>
    from jinja2 import Environment, Markup, FileSystemLoader, Template, select_autoescape
ImportError: cannot import name 'Markup' from 'jinja2' (/home/mauro/workspace/python-env/spid_sp_test/lib/python3.8/site-packages/jinja2/__init__.py)

What can I do?

I'm running Ubuntu 20.04.

peppelinux commented 2 years ago

Install it using a virtualenv, so cryptography will be of that version for sure :) Don't rely on system env

mauromol commented 2 years ago

Yes, as I've written above, I'm using a virtual env!

mauromol commented 2 years ago

Here is the full output when I create the virtual env from scratch, activate it and install spid_sp_test:

mauro@hppb ~/workspace/python-env $ rm -rf spid_sp_test
mauro@hppb ~/workspace/python-env $ python3 -m venv spid_sp_test
mauro@hppb ~/workspace/python-env $ source spid_sp_test/bin/activate
(spid_sp_test) mauro@hppb ~/workspace/python-env $ pip install spid-sp-test --upgrade --no-cache
Collecting spid-sp-test
  Downloading spid_sp_test-1.1.4-py3-none-any.whl (2.5 MB)
     |████████████████████████████████| 2.5 MB 4.5 MB/s 
Collecting xmlschema==1.7.1
  Downloading xmlschema-1.7.1-py3-none-any.whl (256 kB)
     |████████████████████████████████| 256 kB 6.7 MB/s 
Collecting Jinja2>=2.11.3
  Downloading Jinja2-3.1.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 6.8 MB/s 
Collecting pysaml2>=6.5.1
  Downloading pysaml2-7.1.2-py2.py3-none-any.whl (406 kB)
     |████████████████████████████████| 406 kB 6.7 MB/s 
Collecting lxml>=4.6.2
  Downloading lxml-4.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.9 MB)
     |████████████████████████████████| 6.9 MB 5.2 MB/s 
Collecting spid-compliant-certificates>=0.4.1
  Downloading spid_compliant_certificates-0.4.1-py3-none-any.whl (36 kB)
Collecting requests>=2.25.1
  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
     |████████████████████████████████| 63 kB 6.2 MB/s 
Collecting elementpath<3.0.0,>=2.2.2
  Downloading elementpath-2.5.0-py3-none-any.whl (181 kB)
     |████████████████████████████████| 181 kB 6.7 MB/s 
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Requirement already satisfied, skipping upgrade: setuptools in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (44.0.0)
Collecting defusedxml
  Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting pytz
  Downloading pytz-2022.1-py2.py3-none-any.whl (503 kB)
     |████████████████████████████████| 503 kB 5.9 MB/s 
Collecting pyOpenSSL
  Downloading pyOpenSSL-22.0.0-py2.py3-none-any.whl (55 kB)
     |████████████████████████████████| 55 kB 7.2 MB/s 
Collecting cryptography>=1.4
  Downloading cryptography-36.0.2-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.7 MB)
     |████████████████████████████████| 3.7 MB 8.0 MB/s 
Collecting python-dateutil
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |████████████████████████████████| 247 kB 7.0 MB/s 
Collecting importlib-resources; python_version < "3.9"
  Downloading importlib_resources-5.4.0-py3-none-any.whl (28 kB)
Collecting packaging==21.2
  Downloading packaging-21.2-py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 8.0 MB/s 
Collecting ruamel.yaml==0.17.19
  Downloading ruamel.yaml-0.17.19-py3-none-any.whl (109 kB)
     |████████████████████████████████| 109 kB 6.4 MB/s 
Collecting iso3166==2.0.2
  Downloading iso3166-2.0.2-py3-none-any.whl (8.5 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 6.7 MB/s 
Collecting idna<4,>=2.5; python_version >= "3"
  Downloading idna-3.3-py3-none-any.whl (61 kB)
     |████████████████████████████████| 61 kB 6.1 MB/s 
Collecting charset-normalizer~=2.0.0; python_version >= "3"
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
     |████████████████████████████████| 149 kB 6.4 MB/s 
Collecting cffi>=1.12
  Downloading cffi-1.15.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB)
     |████████████████████████████████| 446 kB 6.5 MB/s 
Collecting zipp>=3.1.0; python_version < "3.10"
  Downloading zipp-3.7.0-py3-none-any.whl (5.3 kB)
Collecting pyparsing<3,>=2.0.2
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
     |████████████████████████████████| 67 kB 8.1 MB/s 
Collecting ruamel.yaml.clib>=0.2.6; platform_python_implementation == "CPython" and python_version < "3.11"
  Downloading ruamel.yaml.clib-0.2.6-cp38-cp38-manylinux1_x86_64.whl (570 kB)
     |████████████████████████████████| 570 kB 6.5 MB/s 
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     |████████████████████████████████| 118 kB 6.8 MB/s 
ERROR: spid-compliant-certificates 0.4.1 has requirement cryptography==35.0.0, but you'll have cryptography 36.0.2 which is incompatible.
Installing collected packages: elementpath, xmlschema, MarkupSafe, Jinja2, six, urllib3, idna, charset-normalizer, certifi, requests, defusedxml, pytz, pycparser, cffi, cryptography, pyOpenSSL, python-dateutil, zipp, importlib-resources, pysaml2, lxml, pyparsing, packaging, ruamel.yaml.clib, ruamel.yaml, iso3166, spid-compliant-certificates, spid-sp-test
Successfully installed Jinja2-3.1.0 MarkupSafe-2.1.1 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cryptography-36.0.2 defusedxml-0.7.1 elementpath-2.5.0 idna-3.3 importlib-resources-5.4.0 iso3166-2.0.2 lxml-4.8.0 packaging-21.2 pyOpenSSL-22.0.0 pycparser-2.21 pyparsing-2.4.7 pysaml2-7.1.2 python-dateutil-2.8.2 pytz-2022.1 requests-2.27.1 ruamel.yaml-0.17.19 ruamel.yaml.clib-0.2.6 six-1.16.0 spid-compliant-certificates-0.4.1 spid-sp-test-1.1.4 urllib3-1.26.9 xmlschema-1.7.1 zipp-3.7.0
(spid_sp_test) mauro@hppb ~/workspace/python-env $

Then, I run a second pip install command, without --no-cache: in this way I get the correct version for cryptography:

(spid_sp_test) mauro@hppb ~/workspace/python-env $ pip install spid-sp-test --upgrade
Requirement already up-to-date: spid-sp-test in ./spid_sp_test/lib/python3.8/site-packages (1.1.4)
Requirement already satisfied, skipping upgrade: lxml>=4.6.2 in ./spid_sp_test/lib/python3.8/site-packages (from spid-sp-test) (4.8.0)
Requirement already satisfied, skipping upgrade: spid-compliant-certificates>=0.4.1 in ./spid_sp_test/lib/python3.8/site-packages (from spid-sp-test) (0.4.1)
Requirement already satisfied, skipping upgrade: requests>=2.25.1 in ./spid_sp_test/lib/python3.8/site-packages (from spid-sp-test) (2.27.1)
Requirement already satisfied, skipping upgrade: Jinja2>=2.11.3 in ./spid_sp_test/lib/python3.8/site-packages (from spid-sp-test) (3.1.0)
Requirement already satisfied, skipping upgrade: xmlschema==1.7.1 in ./spid_sp_test/lib/python3.8/site-packages (from spid-sp-test) (1.7.1)
Requirement already satisfied, skipping upgrade: pysaml2>=6.5.1 in ./spid_sp_test/lib/python3.8/site-packages (from spid-sp-test) (7.1.2)
Requirement already satisfied, skipping upgrade: iso3166==2.0.2 in ./spid_sp_test/lib/python3.8/site-packages (from spid-compliant-certificates>=0.4.1->spid-sp-test) (2.0.2)
Requirement already satisfied, skipping upgrade: ruamel.yaml==0.17.19 in ./spid_sp_test/lib/python3.8/site-packages (from spid-compliant-certificates>=0.4.1->spid-sp-test) (0.17.19)
Requirement already satisfied, skipping upgrade: packaging==21.2 in ./spid_sp_test/lib/python3.8/site-packages (from spid-compliant-certificates>=0.4.1->spid-sp-test) (21.2)
Collecting cryptography==35.0.0
  Using cached cryptography-35.0.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.7 MB)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in ./spid_sp_test/lib/python3.8/site-packages (from requests>=2.25.1->spid-sp-test) (2021.10.8)
Requirement already satisfied, skipping upgrade: urllib3<1.27,>=1.21.1 in ./spid_sp_test/lib/python3.8/site-packages (from requests>=2.25.1->spid-sp-test) (1.26.9)
Requirement already satisfied, skipping upgrade: charset-normalizer~=2.0.0; python_version >= "3" in ./spid_sp_test/lib/python3.8/site-packages (from requests>=2.25.1->spid-sp-test) (2.0.12)
Requirement already satisfied, skipping upgrade: idna<4,>=2.5; python_version >= "3" in ./spid_sp_test/lib/python3.8/site-packages (from requests>=2.25.1->spid-sp-test) (3.3)
Requirement already satisfied, skipping upgrade: MarkupSafe>=2.0 in ./spid_sp_test/lib/python3.8/site-packages (from Jinja2>=2.11.3->spid-sp-test) (2.1.1)
Requirement already satisfied, skipping upgrade: elementpath<3.0.0,>=2.2.2 in ./spid_sp_test/lib/python3.8/site-packages (from xmlschema==1.7.1->spid-sp-test) (2.5.0)
Requirement already satisfied, skipping upgrade: defusedxml in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (0.7.1)
Requirement already satisfied, skipping upgrade: python-dateutil in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (2.8.2)
Requirement already satisfied, skipping upgrade: six in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (1.16.0)
Requirement already satisfied, skipping upgrade: pyOpenSSL in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (22.0.0)
Requirement already satisfied, skipping upgrade: importlib-resources; python_version < "3.9" in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (5.4.0)
Requirement already satisfied, skipping upgrade: pytz in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (2022.1)
Requirement already satisfied, skipping upgrade: setuptools in ./spid_sp_test/lib/python3.8/site-packages (from pysaml2>=6.5.1->spid-sp-test) (44.0.0)
Requirement already satisfied, skipping upgrade: ruamel.yaml.clib>=0.2.6; platform_python_implementation == "CPython" and python_version < "3.11" in ./spid_sp_test/lib/python3.8/site-packages (from ruamel.yaml==0.17.19->spid-compliant-certificates>=0.4.1->spid-sp-test) (0.2.6)
Requirement already satisfied, skipping upgrade: pyparsing<3,>=2.0.2 in ./spid_sp_test/lib/python3.8/site-packages (from packaging==21.2->spid-compliant-certificates>=0.4.1->spid-sp-test) (2.4.7)
Requirement already satisfied, skipping upgrade: cffi>=1.12 in ./spid_sp_test/lib/python3.8/site-packages (from cryptography==35.0.0->spid-compliant-certificates>=0.4.1->spid-sp-test) (1.15.0)
Requirement already satisfied, skipping upgrade: zipp>=3.1.0; python_version < "3.10" in ./spid_sp_test/lib/python3.8/site-packages (from importlib-resources; python_version < "3.9"->pysaml2>=6.5.1->spid-sp-test) (3.7.0)
Requirement already satisfied, skipping upgrade: pycparser in ./spid_sp_test/lib/python3.8/site-packages (from cffi>=1.12->cryptography==35.0.0->spid-compliant-certificates>=0.4.1->spid-sp-test) (2.21)
Installing collected packages: cryptography
  Attempting uninstall: cryptography
    Found existing installation: cryptography 36.0.2
    Uninstalling cryptography-36.0.2:
      Successfully uninstalled cryptography-36.0.2
Successfully installed cryptography-35.0.0

Then, I try to start spid_sp_test, but it doesn't work:

(spid_sp_test) mauro@hppb ~/workspace/python-env $ spid_sp_test
Traceback (most recent call last):
  File "/home/mauro/workspace/python-env/spid_sp_test/bin/spid_sp_test", line 17, in <module>
    from spid_sp_test.response import SpidSpResponseCheck
  File "/home/mauro/workspace/python-env/spid_sp_test/lib/python3.8/site-packages/spid_sp_test/response.py", line 14, in <module>
    from jinja2 import Environment, Markup, FileSystemLoader, Template, select_autoescape
ImportError: cannot import name 'Markup' from 'jinja2' (/home/mauro/workspace/python-env/spid_sp_test/lib/python3.8/site-packages/jinja2/__init__.py)
peppelinux commented 2 years ago

IT seems a breaking change of the last release of jinja2

https://github.com/pallets/jinja/issues?q=is%3Aissue+ImportError%3A+cannot+import+name+%27Markup%27+from+%27jinja2%27+is%3Aclosed

https://github.com/pallets/jinja/issues/1640

here the changelog https://github.com/pallets/jinja/blob/6f79daafe868b8ef20baecdcb45d12299529ed2e/CHANGES.rst#version-310

please try to edit /home/mauro/workspace/python-env/spid_sp_test/lib/python3.8/site-packages/spid_sp_test/response.pyand replace from jinja2 import Environment, Markup, FileSystemLoader, Template, select_autoescape with 

from jinja2 import Environment, FileSystemLoader, Template, select_autoescape
from markupsafe import Markup

save, run, and let me know. We can do a new release with a new version of jinja2

peppelinux commented 2 years ago

@mauromol Done here: https://github.com/italia/spid-sp-test/commit/88138bf4401ffbb8ed74bd47203b9b0debe3fd28

if your test will be fine we'll do a release, the v1.1.5

mauromol commented 2 years ago

Version 1.1.5 fixes the jinja problem for me, thank you!