Because aws-sso-util (4.32.0) depends on jsonschema (>=3.2.0,<4.0.0)

[craigbaldacchino-wel]

Dependencies for jsonschema are help back "Because aws-sso-util (4.32.0) depends on jsonschema (>=3.2.0,<4.0.0)"

jsonschema is up to 4.20.0 - https://pypi.org/project/jsonschema/

Would there be anything specific holding back this being updated?

[craigbaldacchino-wel]

May not be worth the solve due to compatibility with python 3.7, jsonschema 4.20.0 is compatible with python >=3.8

The current project's Python requirement (>=3.7,<4.0) is not compatible with some of the required packages Python requirement:

• jsonschema requires Python >=3.8, so it will not be satisfied for Python >=3.7,<3.8

Because no versions of jsonschema match >4.20.0,<5.0.0 and jsonschema (4.20.0) requires Python >=3.8, jsonschema is forbidden.

So, because aws-sso-util depends on jsonschema (^4.20.0), version solving failed.

BUT can probably update jsonschema (3.2.0 -> 4.17.3)

[huy-nguyen26]

Are there any updates on this issue, please? I've been stuck on this as well! Thank you

[iainelder]

@huy-nguyen26, why does the jsonschema version matter to you?

Did you install aws-sso-util in the system Python environment or your default user Python environment, and now you see a conflict with another application?

The Quickstart guide shows how to avoid such conflicts by installing using pipx.

pipx install aws-sso-util

[huy-nguyen26]

@iainelder yes, I also tried pipx but I have the same error. Am I missing something please?

[iainelder]

I've never seen that error from pipx.

I think something in your system's environment causes it.

I need more information to reproduce it.

Your screenshot shows a file path that ends /pipx/cmd_2024-03-28_22.10.08_pip_errors.log.

That file contains the full output of pip. Can you share the content here?

Which versions of Python, pip, and pipx are you using?

---

To isolate the problem from my own system's environment I will try to reproduce your error in a Docker container.

I started the container like this.

docker run -it python:3.12 bash

I used these commands to install and configure pip and pipx before installing aws-sso-util.

export PIP_ROOT_USER_ACTION=ignore 
PATH="$PATH:/root/.local/bin"
pip install --upgrade pip --quiet
pip install pipx --quiet
pipx install aws-sso-util

I got this output, which means aws-sso-util installed successfully.

  installed package aws-sso-util 4.32.0, installed using Python 3.12.2
  These apps are now globally available
    - aws-sso-util
done! ✨ 🌟 ✨

I found the command log in the folder /root/.local/state/pipx/log/.

# ls /root/.local/state/pipx/log/
cmd_2024-03-30_19.33.32.log

I use this command to write out the content of the file. The * makes it work for any date and time in the file name.

cat /root/.local/state/pipx/log/cmd_*.log

   116.9ms (setup:922): 2024-03-30 19:33:32
   117.0ms (setup:923): /usr/local/bin/pipx install aws-sso-util
   117.0ms (setup:924): pipx version is 1.5.0
   117.1ms (setup:925): Default python interpreter is '/usr/local/bin/python'
   117.2ms (mkdir:75): creating directory /root/.local/share/pipx/venvs
   117.4ms (mkdir:75): creating directory /root/.local/bin
   117.5ms (mkdir:75): creating directory /root/.local/share/man
   117.6ms (mkdir:75): creating directory /root/.cache/pipx
   117.8ms (mkdir:75): creating directory /root/.local/share/pipx/py
   118.0ms (setup:939): Adding CACHEDIR.TAG to cache directory
   118.1ms (setup:939): Adding CACHEDIR.TAG to cache directory
   119.3ms (package_name_from_spec:379): Determined package name: aws-sso-util
   119.3ms (package_name_from_spec:380): Package name determined in 0.0s
   119.8ms (run_subprocess:174): running /usr/local/bin/python -m venv --without-pip /root/.local/share/pipx/venvs/aws-sso-util
   171.4ms (run_subprocess:194): stdout:
   171.5ms (run_subprocess:196): stderr:
   171.5ms (run_subprocess:197): returncode: 0
   171.9ms (run_subprocess:174): running /usr/local/bin/python -m venv --clear /root/.local/share/pipx/shared
  3277.3ms (run_subprocess:194): stdout:
  3277.4ms (run_subprocess:196): stderr:
  3277.4ms (run_subprocess:197): returncode: 0
  3277.7ms (run_subprocess:174): running <checking pip's availability>
  3295.3ms (run_subprocess:194): stdout: ModuleSpec(name='pip', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f0574046780>, origin='/root/.local/share/pipx/shared/lib/python3.12/site-packages/pip/__init__.py', submodule_search_locations=['/root/.local/share/pipx/shared/lib/python3.12/site-packages/pip'])
  3295.3ms (run_subprocess:197): returncode: 0
  3295.4ms (upgrade:103): Upgrading shared libraries in /root/.local/share/pipx/shared
  3295.7ms (run_subprocess:174): running /root/.local/share/pipx/shared/bin/python -m pip --no-input --disable-pip-version-check install --force-reinstall -q --upgrade pip >= 23.1
  5990.9ms (run_subprocess:194): stdout:
  5991.0ms (run_subprocess:196): stderr:
  5991.0ms (run_subprocess:197): returncode: 0
  5991.3ms (run_subprocess:174): running /root/.local/share/pipx/venvs/aws-sso-util/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
  6016.9ms (run_subprocess:194): stdout: /root/.local/share/pipx/venvs/aws-sso-util/lib/python3.12/site-packages
  6017.0ms (run_subprocess:197): returncode: 0
  6017.2ms (run_subprocess:174): running /root/.local/share/pipx/shared/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
  6042.2ms (run_subprocess:194): stdout: /root/.local/share/pipx/shared/lib/python3.12/site-packages
  6042.3ms (run_subprocess:197): returncode: 0
  6042.6ms (run_subprocess:174): running /root/.local/share/pipx/venvs/aws-sso-util/bin/python --version
  6045.6ms (run_subprocess:194): stdout: Python 3.12.2
  6045.6ms (run_subprocess:196): stderr:
  6045.7ms (run_subprocess:197): returncode: 0
  6046.0ms (_parsed_package_to_package_or_url:137): cleaned package spec: aws-sso-util
  6046.3ms (run_subprocess:174): running /root/.local/share/pipx/venvs/aws-sso-util/bin/python -m pip --no-input install aws-sso-util
 19189.8ms (run_subprocess:197): returncode: 0
 19190.3ms (run_subprocess:174): running <fetch_info_in_venv commands>
 19223.0ms (run_subprocess:194): stdout: {"sys_path": ["/usr/local/lib/python312.zip", "/usr/local/lib/python3.12", "/usr/local/lib/python3.12/lib-dynload", "/root/.local/share/pipx/venvs/aws-sso-util/lib/python3.12/site-packages", "/root/.local/share/pipx/shared/lib/python3.12/site-packages"], "python_version": "3.12.2", "environment": {"implementation_name": "cpython", "implementation_version": "3.12.2", "os_name": "posix", "platform_machine": "x86_64", "platform_release": "5.15.0-101-generic", "platform_system": "Linux", "platform_version": "#111~20.04.1-Ubuntu SMP Mon Mar 11 15:44:43 UTC 2024", "python_full_version": "3.12.2", "platform_python_implementation": "CPython", "python_version": "3.12", "sys_platform": "linux"}}
 19223.1ms (run_subprocess:197): returncode: 0
 19448.0ms (get_venv_metadata_for_package:348): get_venv_metadata_for_package: 258ms
 19448.1ms (_parsed_package_to_package_or_url:137): cleaned package spec: aws-sso-util
 19450.4ms (cli:1001): pipx finished.

(I'm no pip expert so I don't know what every log line means. But maybe yours will show something to help.)

The components in the container have these versions:

• python --version: Python 3.12.2
• pip --version: pip 24.0 from /usr/local/lib/python3.12/site-packages/pip (python 3.12)
• pipx --version: 1.5.0
• aws-sso-util --version: 4.32.0

Docker and the image have these versions:

• docker --version: Docker version 25.0.3, build 4debf41
• docker images python:3.12 --quiet: 6cbe1053f244

[huy-nguyen26]

@iainelder , the content of my log is

PIP STDOUT
----------
Looking in indexes: https://pypi.org/project
Collecting aws-sso-util
  Using cached aws_sso_util-4.32.0-py3-none-any.whl (62 kB)
Collecting aws-error-utils<3.0,>=2.4 (from aws-sso-util)
  Using cached aws_error_utils-2.7.0-py3-none-any.whl (11 kB)
Collecting aws-sso-lib<2.0.0,>=1.13.0 (from aws-sso-util)
  Using cached aws_sso_lib-1.14.0-py3-none-any.whl (33 kB)
Collecting boto3<2.0.0,>=1.24.60 (from aws-sso-util)
  Using cached boto3-1.34.73-py3-none-any.whl (139 kB)
Collecting click<9.0.0,>=8.0.0 (from aws-sso-util)
  Downloading click-8.1.7-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 1.9 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of aws-sso-util to determine which version is compatible with other requirements. This could take a while.

PIP STDERR
----------
ERROR: Could not find a version that satisfies the requirement jsonschema<4.0.0,>=3.2.0 (from aws-sso-util) (from versions: 4.21.1)
ERROR: No matching distribution found for jsonschema<4.0.0,>=3.2.0

My version of

• pipx: 1.4.3
• pip: 24.0
• python: 3.12.2

[iainelder]

@huy-nguyen26, thanks for sharing your log content and your component versions.

---

Your pipx version is 1.4.3.

The latest pipx version is 1.5.0, released 4 days ago.

Can you upgrade your pipx to the latest version and try again?

Nothing seems obvious from the pipx changelog, but maybe the newer version solves the problem.

Please try that before we investigate anything from the 1.4.3 log.

---

Two things stand out from the 1.4.3 log.

As you already pointed out, the log's final message is the error about the jsonschema package.

ERROR: Could not find a version that satisfies the requirement jsonschema<4.0.0,>=3.2.0 (from aws-sso-util) (from versions: 4.21.1)
ERROR: No matching distribution found for jsonschema<4.0.0,>=3.2.0

It also says that "pip is looking at multiple version of aws-sso-util". That looks unusual.

INFO: pip is looking at multiple versions of aws-sso-util to determine which version is compatible with other requirements. This could take a while.

I would need to research both of those messages. Please try with pipx 1.5.0 before we spend time researching.

[huy-nguyen26]

@iainelder , yes I upgraded my pipx to 1.5.0 and I still have the same problem.

❯ pipx install aws-sso-util
⚠️ Found a space in the home path. We heavily discourage this, due to multiple incompatibilities. Please check our docs for more information on this, as
    well as some pointers on how to migrate to a different home path.
Fatal error from pip prevented installation. Full pip output in file:
    /Users/.../.../.../cmd_2024-04-03_11.48.27_pip_errors.log

pip seemed to fail to build package:
    click<9.0.0,>=8.0.0

Some possibly relevant errors from pip install:
    ERROR: Could not find a version that satisfies the requirement jsonschema<4.0.0,>=3.2.0 (from aws-sso-util) (from versions: 4.21.1)
    ERROR: No matching distribution found for jsonschema<4.0.0,>=3.2.0

Error installing aws-sso-util.

And inside the error log

PIP STDOUT
----------
Looking in indexes: https://pypi.org/project
Collecting aws-sso-util
  Using cached aws_sso_util-4.32.0-py3-none-any.whl (62 kB)
Collecting aws-error-utils<3.0,>=2.4 (from aws-sso-util)
  Using cached aws_error_utils-2.7.0-py3-none-any.whl (11 kB)
Collecting aws-sso-lib<2.0.0,>=1.13.0 (from aws-sso-util)
  Using cached aws_sso_lib-1.14.0-py3-none-any.whl (33 kB)
Collecting boto3<2.0.0,>=1.24.60 (from aws-sso-util)
  Downloading boto3-1.34.76-py3-none-any.whl (139 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.3/139.3 kB 4.9 MB/s eta 0:00:00
Collecting click<9.0.0,>=8.0.0 (from aws-sso-util)
  Using cached click-8.1.7-py3-none-any.whl (97 kB)
INFO: pip is looking at multiple versions of aws-sso-util to determine which version is compatible with other requirements. This could take a while.

PIP STDERR
----------
ERROR: Could not find a version that satisfies the requirement jsonschema<4.0.0,>=3.2.0 (from aws-sso-util) (from versions: 4.21.1)
ERROR: No matching distribution found for jsonschema<4.0.0,>=3.2.0

One thing I noticed that when I run pipx --version, the output is as follow

⚠️ Found a space in the home path. We heavily discourage this, due to multiple incompatibilities. Please check our docs for more information on this, as
    well as some pointers on how to migrate to a different home path.
1.5.0

I'm unsure why I'm receiving this warning. I'm not certain if it's relevant or not ?!

[huy-nguyen26]

@iainelder I tried to install aws-sso-util through brew install and it works !! I think I will do more tests on it

[iainelder]

I tried to install aws-sso-util through brew install and it works

@huy-nguyen26, that's great news!

I see Homebrew has a formula for aws-sso-util.

• formulae.brew.sh
• aws-sso-util.rb on GitHub

I think brew may install and configure its own Python environment for anything that depends on Python. I think that's how Homebrew avoids whatever problem exists in your first environment.

---

I think I will do more tests on it

Do you still want support for the earlier errors?

⚠️ Found a space in the home path.

I can reproduce the message in Docker. I run the same installation commands. Before I run those commands I set PIPX_HOME to a path that contains a space.

export PIPX_HOME=~/Contains\ Space 
mkdir "$PIPX_HOME"
export PIP_ROOT_USER_ACTION=ignore 
PATH="$PATH:/root/.local/bin"
pip install --upgrade pip --quiet
pip install pipx --quiet
pipx install aws-sso-util

I don't think the space in the name causes the jsonschema error because aws-sso-util still gets installed.

⚠️ Found a space in the home path. We heavily discourage this, due to multiple incompatibilities. Please check our
    docs for more information on this, as well as some pointers on how to migrate to a different home path.
  installed package aws-sso-util 4.32.0, installed using Python 3.12.2
  These apps are now globally available
    - aws-sso-util
done! ✨ 🌟 ✨

[iainelder]

@huy-nguyen26 while researching PIPX_HOME I found issue 1198 in the pipx project on GitHub.

It appears that since release 1.3.0, when installing pipx on macOS Ventura, it chooses to set $PIPX_HOME to ~/Library/Application Support/pipx

The issue explains why the "space in home path" message appears and suggests workarounds. I didn't understand everything there. Maybe it will mean more to you as a macOS user.

[iainelder]

@huy-nguyen26, what do you see when you run pip index versions jsonschema in your first environment?

I ask because this error bothers me:

ERROR: No matching distribution found for jsonschema<4.0.0,>=3.2.0

The jsonschema<4.0.0,>=3.2.0 requirement comes from pyproject.toml.

jsonschema = "^3.2.0"

On my system pipx finds version 3.2.0 to satisfy the requirement.

I don't understand why your system doesn't find version 3.2.0.

---

I use the following pip command to query PyPI for all available jsonschema versions.

I pipe it through fold because GitHub code blocks don't wrap long lines.

pip index versions jsonschema | fold -s -w 80

jsonschema (4.21.1)
Available versions: 4.21.1, 4.21.0, 4.20.0, 4.19.2, 4.19.1, 4.19.0, 4.18.6, 
4.18.5, 4.18.4, 4.18.3, 4.18.2, 4.18.1, 4.18.0, 4.17.3, 4.17.1, 4.17.0, 4.16.0, 
4.15.0, 4.14.0, 4.13.0, 4.12.1, 4.12.0, 4.11.0, 4.10.3, 4.10.2, 4.10.1, 4.10.0, 
4.9.1, 4.9.0, 4.8.0, 4.7.2, 4.7.1, 4.7.0, 4.6.2, 4.6.1, 4.6.0, 4.5.1, 4.5.0, 
4.4.0, 4.3.3, 4.3.2, 4.3.1, 4.3.0, 4.2.1, 4.2.0, 4.1.2, 4.1.1, 4.1.0, 4.0.1, 
3.2.0, 3.1.1, 3.1.0, 3.0.2, 3.0.1, 3.0.0, 2.6.0, 2.5.1, 2.5.0, 2.4.0, 2.3.0, 
2.2.0, 2.1.0, 2.0.0, 1.3.0, 1.2.0, 1.1.0, 1.0.0, 0.8.0, 0.7, 0.6, 0.5, 0.4, 
0.3, 0.2

The list contains a version 3.2.0.

Version 3.2.0 is the only version that satisfies the requirement.

[sandangel]

Hello, can we update the dependencies? I use aws-sso-util python code in my project (not CLI) so it's conflicting dependencies with my jupyter notebook. :(

[iainelder]

@sandangel , I haven't used aws-sso-util in a Jupyter notebook. I don't know if the author expected that use case.

Can you create a new GitHub issue with the complete error describing the conflict and a way to reproduce it?

Ping me on that issue and I will help you to find a solution.

[sandangel]

Hi, I have a project, with dev dependencies are aws-sso-utils and jupyterlab. Because of the version constraint you have in aws-sso-utils for jsonschema, I can not install jupyterlab in my project. So the issue is the same with this issue. Can we update the dependency?

[iainelder]

@sandangel , I'm not the repo owner so I can't update the dependency for you.

If you give me instructions to reproduce your error, I may be able to help you find a workaround.

I'm unfamiliar with Jupyter so I will need some help from you.

Can you describe your installation steps and the error you see?

[sandangel]

Hi, you just need to install both at the same time in same venv and it will show the error. It’s pretty common error for dependencies

[benkehoe]

All right folks, I know I've been absent for a long time, but I'm back. It will take a bit for me to get spun back up on my projects, but I will try to get this done as the first thing.

[iainelder]

Hooray, Ben's back!

@benkehoe, what is the solution here? Upgrade jsonschema? Relax the constraint? Split the CloudFormation support into a new package? (#107)

---

@sandangel, are you using the CloudFormation features of aws-sso-util?

If not, then I think you can safely ignore the error. (How you ignore the error in practice depends on how you install the dependencies.)

aws-sso-util uses jsonschema only for its CloudFormation support (README, config.py).

I don't know how jupyterlab uses jsonschema, but I would favor its version to be safe.

If you set up your virtualenv using pip, first install aws-sso-util, then install jupyterlab, and you should end up with a version of jsonschema that is compatible with jupyterlab. (See test 1 below for an example.)

---

Notes for testing the jsonschema version behavior.

Write helper.sh. It creates a helper function to install a package quietly and another to install a list of packages in order and check the final version of jsonschema.

pip_install() {
    PIP_ROOT_USER_ACTION=ignore pip install --quiet "$@"
}

install_all_and_check_jsonschema() {
    for package in "$@"; do
        pip_install "$package"
    done
    pip freeze | grep -P '^jsonschema=='
}

Make another helper function in the main shell to test a command in the Python 3.12 Docker container.

test_in_docker() {
   docker run -it -v "$PWD/helper.sh:/helper.sh" python:3.12 bash --rcfile /helper.sh -i -c "$*"
}

Test 1: install aws-sso-util, then jupyterlab.

test_in_docker install_all_and_check_jsonschema aws-sso-util jupyterlab

The output shows that the final jsonschema version is 4.22.0, which is required by jupyterlab and incompatible with aws-sso-util.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
aws-sso-util 4.32.0 requires jsonschema<4.0.0,>=3.2.0, but you have jsonschema 4.22.0 which is incompatible.
jsonschema==4.22.0

Test 2: install jupyterlab, then aws-sso-util.

test_in_docker install_all_and_check_jsonschema jupyterlab aws-sso-util

The output shows that the final jsonschema version is 3.2.0, which is required by aws-sso-util and incompatible with jupyterlab.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
jupyter-events 0.10.0 requires jsonschema[format-nongpl]>=4.18.0, but you have jsonschema 3.2.0 which is incompatible.
jupyterlab-server 2.27.2 requires jsonschema>=4.18.0, but you have jsonschema 3.2.0 which is incompatible.
jsonschema==3.2.0

[sandangel]

I don't use cloudformation. maybe we can set cloudformation related packages to optional dependencies then?

[benkehoe]

From the jsonschema changelog, I don't think the major version change has any consequences for the aws-sso-util code that's using it, and I can just update the dependency to ^4.0.0, which will keep it working for Python 3.7. A later change can deprecate 3.7 (and 3.8, which is coming up too).

But, to make sure of that, I'd like to test the CloudFormation functionality, and unfortunately my CloudFormation tests are currently broken and I'm not sure how long it will take to get them working. Moving that functionality out of aws-sso-util would require a major version bump, which I don't want to do.

Let me see if I can throw some manual tests together quickly to validate it.

[benkehoe]

Ok, I released this as v4.33. Please test and let me know if it's all fixed for you.

[sandangel]

nice, it fixed the issue. thank you so much.