Open JoeDupuis opened 3 years ago
It seems there is only a single $PYTHONPATH and ordering matters.
https://github.com/NixOS/nixpkgs/issues/10597#issuecomment-151461850
$ nix-shell -p aws-sam-cli awscli2 --run 'echo $PYTHONPATH'
Unsetting PYTHONPATH
appears to solve this issue:
$ nix-shell aws-sam-cli awscli2 --run 'unset PYTHONPATH; aws --version'
aws-cli/2.1.17 Python/3.8.6 Linux/5.10.5 source/x86_64.nixos.21 prompt/off
same issue :(
⋊> nix-shell -I nixpkgs=https://github.com/nixos/nixpkgs/archive/e14843556cb4ed2286c6752a32abf85beec0e19c.tar.gz -p python39 python39Packages.boto3 awscli2 --run "aws --version"
aws-cli/2.2.4 Python/3.8.9 Darwin/20.5.0 source/x86_64 prompt/off
⋊> nix-shell -I nixpkgs=https://github.com/nixos/nixpkgs/archive/e14843556cb4ed2286c6752a32abf85beec0e19c.tar.gz -p awscli2 python39 python39Packages.boto3 --run "aws --version"
cannot import name 'SSOTokenFetcher' from 'botocore.utils' (/nix/store/w2vpzhgiam1jh5r73z15m34gjhf0m1z7-python3.9-botocore-1.20.52/lib/python3.9/site-packages/botocore/utils.py)
nix-info:
⋊> nix-shell -p nix-info --run "nix-info -m"
- system: `"x86_64-darwin"`
- host os: `Darwin 20.5.0, macOS 10.16`
- multi-user?: `no`
- sandbox: `no`
- version: `nix-env (Nix) 2.4pre20210503_6d2553a`
- channels(peter.romfeld): `"nixpkgs-21.11pre292293.84aa23742f6"`
- nixpkgs: `/Users/peter.romfeld/.nix-defexpr/channels/nixpkgs`
Hi, I met the same issues, if somebody needs fast fix, try to remove boto3 and botocore from pip repository.
I'm seeing a similar issue when combining aws-adfs
with awscli2
:
± nix-shell -p awscli2 -p python38Packages.aws-adfs
$ aws
...
ImportError: cannot import name 'resolve_imds_endpoint_mode' from 'botocore.utils'
Quickfix for me:
pkgs.writeScriptBin "aws" ''
#! /usr/bin/env nix-shell
#! nix-shell --pure -i bash -p awscli2
exec aws "$@"
''
Quickfix for me:
pkgs.writeScriptBin "aws" '' #! /usr/bin/env nix-shell #! nix-shell --pure -i bash -p awscli2 exec aws "$@" ''
I tried this while using aws-vault to assume roles, and got credentials errors, so I assume that aws-vault uses environment variables, which are blocked by
nix-shell --pure
. (Just as a warning to others because I wasted a few hours on this 😅)
I marked this as stale due to inactivity. → More info
I'm not seeing this anymore.
Tested with nixpkgs at 3ad64d9e2d5
, not on nixos.
This sounds like an issue I've seen with using awscli2
and python39
in the same mkShell
call (on commit c8bc2f2c0d4abafe71132097976380e59d14c7c8).
e.g.
# ...
with nixpkgs;
mkShell {
buildInputs = [
awscli2
python39
];
}
The observed result is that Python 3.10 is resolved in which python3
because awscli2
comes first. The workaround used in my case was to force awscli2
to use the same Python version.
# ...
with nixpkgs;
mkShell {
buildInputs = [
(awscli2.override { python3 = python39; })
python39
];
}
Ideally this "PATH
leakage" wouldn't be occurring at all. As others have pointed out, PYTHONPATH
is also involved with the specific issue reported.
I just ran into what I believe is the same thing with ansible
and awscli2
. Having ansible
listed first in mkShell causes aws to barf.
nix-shell -p ansible -p awscli2 --command aws
Traceback (most recent call last):
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/bin/.aws-wrapped", line 20, in <module>
import awscli.clidriver
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/clidriver.py", line 21, in <module>
import botocore.session
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/session.py", line 27, in <module>
import botocore.client
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/client.py", line 16, in <module>
from botocore import UNSIGNED, waiter, xform_name
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/waiter.py", line 17, in <module>
from botocore.docs.docstring import WaiterDocstring
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/__init__.py", line 15, in <module>
from botocore.docs.service import ServiceDocumenter
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/service.py", line 14, in <module>
from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/client.py", line 16, in <module>
from botocore.docs.example import ResponseExampleDocumenter
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/example.py", line 13, in <module>
from botocore.docs.shape import ShapeDocumenter
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/shape.py", line 19, in <module>
from botocore.utils import is_json_value_header
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/utils.py", line 35, in <module>
import botocore.httpsession
File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/httpsession.py", line 21, in <module>
from urllib3.util.ssl_ import (
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/nix/store/ram7ivmskyhmanjdkx92l408y8x7facd-python3.11-urllib3-2.0.7/lib/python3.11/site-packages/urllib3/util/ssl_.py)
Switching the order fixes aws but probably upsets ansible.
nix-shell -p awscli2 -p ansible --command aws
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: the following arguments are required: command
What's strange to me is that the urllib3
path from the error is for the wrong version but the path in the wrapper is correct. I don't have enough python / nix knowledge to speculate or postulate why that might be.
Describe the bug There is a clash with the version of awscli2 and aws-sam-cli Depending on the order they are loaded in the environment awscli2 won't run.
It looks like both will load a version of botocore on the path and one version is not compatible with the other.
botocore seems is not a direct dependency of aws-sam-cli, but a sub-dependency.
Still investigating for more details.
To Reproduce
nix-shell -p aws-sam-cli awscli2 --run "aws --version"
Will output :Expected behavior Flipping the order:
nix-shell -p awscli2 aws-sam-cli --run "aws --version"
output the version.SAM cli works in both case.
Additional context I tested with various nixpkgs on unstable on osx. I can't test the latest unstable on osx yet. I'll test it on linux later tonight if I don't find the fix.
Notify maintainers
aws-sam-cli: @andreabedini @lo1tuma
awscli2: @bhipple @davegallant
Metadata Please run
nix-shell -p nix-info --run "nix-info -m"
and paste the result.I ran with:
and this pin:
https://github.com/nixos/nixpkgs/archive/bdbcfb436060f46409a585c78335b1f75646695b.tar.gz
Maintainer information: