Build failed - Error: PythonPipBuilder:ResolveDependencies

[fniko]

Description

Trying to create Python Lambda function from PyCharm not working

Steps to reproduce

Invoke Lambda creation using "Create new AWS Lambda..." Runtime: Python 3.7 requirements.txt: https://pastebin.com/HpU1it63 Run using strict template file: template.yml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An AWS Serverless Specification template describing your function.
Resources:
  test3:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: order.create_order
      Runtime: python3.7
      CodeUri: .
      Description: ''
      MemorySize: 128
      Timeout: 300
      Role: 'arn:aws:iam::123456789:role/service-role/test-role'
      VpcConfig:
        SecurityGroupIds:
          - sg-62e1c42f
        SubnetIds:
          - subnet-26e9c26e
          - subnet-68a3c632
          - subnet-b8edb5de

Observed result

Used graphical create, not CLI, error:

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {msgpack-python==0.5.6(sdist), pyrsistent==0.15.7(sdist), simplejson==3.17.0(wheel), pyyaml==5.3(sdist)}

Full log: https://pastebin.com/asgCa9D4

Expected result

Successful build

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macOS Mojave 10.14.6 SAM: SAM CLI, version 0.43.0 P: Python 3.7.6 IDE: PyCharm 2019.3 AWS: aws-cli/2.0.1 Python/3.7.4 Darwin/18.7.0 botocore/2.0.0dev5 used wheel: wheel 0.34.2 used pip: pip 19.0.3

[fniko]

I solved this using sam build --use-container, however this works only when template.yml file is present. Using the graphical form does not allow to insert --use-container and it fails, so I do not count this as a solution, but workaround instead... // Also since I am unable to build app locally in my IDE, I am unable to run/debug it, so it makes no sense to use this workaround.

[c2tarun]

Are you using AWS Toolkit for Pycharm?

[dperconti]

I'm seeing something similar:

$ sam build
Building resource 'PaymentsDataStreamFunction'
Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pyyaml==5.3.1(sdist), pyrsistent==0.15.7(sdist)}

Both pyyaml & pyrsistent are inside my requirements.txt file, which have been installed inside a virtualenv.

[sriram-mv]

Do you have wheel installed? if not sam build --use-container may be the best bet.

[dperconti]

wheel is installed. --use-container also fails with a similar error but with different deps. I tried creating a brand new project which worked fine. So, it's probably something within my project that is messed up.

[fniko]

@c2tarun yes, I am using this plugin

[heitorlessa]

Same here - Used another dep that depends on jsonschema and bumped into this. Can't use --use-container as I have a shared dep two directories up. For that to work I'd need to copy the wheel to every service I have, or have a way to mount a volume during sam build.

Thanks for opening the issue, I'll spend a couple of days rewriting the lib and build a wheel from that instead

[emulvihill]

I'm getting same issue when I use sam build --use-container (Windows, Python 3.6, SAM CLI version 0.47.0): Build Failed Error: PythonPipBuilder:ResolveDependencies - {tensorflow==1.15.0(wheel), grpcio==1.28.1(wheel)}

[guywilsonjr]

Same. It seems like it fails to build anything with tensorflow or better yet grpcio

[viotti]

My current workaround is to manually add grpcio to the requirements.txt file, but fixed to an older version.

grpcio==1.27.2

It seems to fail with version 1.28.

[deanrabinowitz4]

I'm seeing a similar error with the following in requirements.txt pyorc==0.2.0

I am running:

sam build --use-container

Output: Starting Build inside a container Building resource 'CdcFunction'

Fetching lambci/lambda:build-python3.7 Docker container image...... Mounting /path/to/my/app as /tmp/samcli/source:ro,delegated inside runtime container

Build Failed Running PythonPipBuilder:ResolveDependencies Error: PythonPipBuilder:ResolveDependencies - {pyorc==0.2.0(wheel)}

wheel is installed. I'm getting the same error when I add psycopg2 to the requirements. Here is my template.yaml for reference

Globals:
  Function:
    Timeout: 180

Resources:
  CdcFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: lambda/
      Handler: lambda.lambda_handler
      Runtime: python3.7

[mooperd]

+1

[carpnick]

+1

Error: PythonPipBuilder:ResolveDependencies - {pyrsistent==0.16.0(sdist), pyyaml==5.1.2(sdist)}

[sriram-mv]

If your dependencies have C bindings involved, building within the container is the way forward in those cases. Toolkits have an explicit checkbox that can be enabled to allow for all builds to be within a build container.

Closing this issue, please re-open if you feel this doesn't answer your original question.

[etishka]

got the following error started appearing during sam build

Error: PythonPipBuilder:ResolveDependencies - {protobuf==3.17.1(wheel)}

SAM CLI, version 1.20.0 Python 3.8.7 pip 21.1.2

--use-container doesn't solve the problem.

[greensky-code]

UnsatisfiedLinkError: com.amazonaws.kinesisvideo.producer.jni.NativeKinesisVideoProducerJni.getNativeLibraryVersion()Ljava/lang/String I am following this workspace https://github.com/backdoorcodr/amazon-kinesis-video-streams-producer-sdk-java Expectation is to run kinesis video stream using webcam in java. Please help

[leonardishere]

I was running into Error: PythonPipBuilder:ResolveDependencies - {cytoolz==0.11.0(wheel)} with sam 0.45.0. Upgrading sam to latest (1.27.2) solved it

[CGarces]

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pillow==8.3.1(wheel)}

Solved updating SAM CLI from version 1.22.0 to version 1.29.0

[otmezger]

I'm having the same issue

sam build --use-container --template ./my_template.yaml --debug

produces

Error: PythonPipBuilder:ResolveDependencies - Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-download-ufgf7ow5/fastparquet/

However, installing the requirements.txt with pip in the host machine produces no error.

I also tried to update my sam version from 1.27.2 -> 1.31.0 and I get the same error.

[dslove]

I have the same issue. Tried installing wheel, using --use-container, updating sam to 1.31. None is helpful.

The most weird thing is it was working last week, but suddenly broke today. Nothing has changed at all!

[hoffa]

Reopening since this still seems to be an issue. Potentially related: https://github.com/aws/aws-lambda-builders/issues/280

[ronitsuz]

+1 Error: PythonPipBuilder:ResolveDependencies - {olefile==0.46(sdist), future==0.18.2(sdist), wrapt==1.12.1(sdist)}

[ryanaklein]

I was having a similar problem just running from the command line with SAM cli 1.15. A little behind, I know!

Here's what I had before using a python pipenv virtual environment.

OS: macOS Monterey 12.0.1 SAM: SAM CLI, version 1.15.0 P: Python 3.8.7 IDE: 1.61.2 AWS: aws-cli/2.1.21 Python/3.7.4 Darwin/21.1.0 exe/x86_64 prompt/off used wheel: wheel 0.37.0 used pip:21.3.1

I upgraded to 1.34 and build seems to work.

[ronitsuz]

Thanks @ryanaklein ! Unfortunately in my case it's already version 1.34.1, though the other versions are different (for instance, python 3.9).

[jojo786]

My environment:

• macOS Catalina
• Python 3.9.7
• pip 21.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)
• SAM CLI, version 1.35.0

I am building this project: https://github.com/jojo786/awswhatsnew, which works perfectly locally, but not with sam.

If i do a sam build without a requirements.txt, its successful. But with this requirements.txt file of:

feedparser
requests
python-twitter
aws_lambda_powertools

I get this when doing a sam build

Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {wrapt==1.13.3(wheel)}

These are the things I have tried unsuccessfully:

• start with a new clean venv, and install all dependencies
• use sam build with --use-container
• install wheel as mentioned in https://github.com/aws/aws-lambda-builders/issues/280
• include wheel in requirements.txt
• download wrapt wheel manually and install

Linked issues:

• https://github.com/aws/aws-sam-cli/issues/1380
• https://github.com/aws/aws-lambda-builders/issues/208
• https://github.com/aws/aws-sam-cli/issues/2291
• https://github.com/aws/aws-lambda-builders/issues/201

[leryss]

This fixed the ResolveDependencies error for me pip install wheel

[tim-blackmore]

I am also seeing a problem. I can deploy to AWS without anything in my requirements file, but when I add boto3 (for instance to my requirements file I get the following error)

Build Failed
Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

My requirements.txt looks like this...

boto3==1.18.0 botocore==1.21.65 jmespath==0.10.0 python-dateutil==2.8.2 s3transfer==0.5.0 six==1.16.0 urllib3==1.26.7

I've tried upgrading pip and wheel but this has not helped.

[awood45]

This definitely seems to be some sort of pip issue around string encoding, lots of results like this: https://stackoverflow.com/questions/41906206/pip-unicodedecodeerror-utf8-codec-cant-decode-byte

I'm unfortunately not sure what we can do about this, we're reliant on pip for Python builds, and this doesn't seem to come from any pass-through behavior. Does it behave differently if you run pip directly?

[tim-blackmore]

Thanks for coming back to me. You are right it was an encoding issue of the requirements.txt file. To check it I opened it and printed the encoding info.

f=open('requirements.txt, 'w') print(f.encoding)

The result was cp1252.

I altered this by changing it in the Windows control panel

Control panel (make sure you are set to category view)> Clock and Region > Region > Adminsitrative > Change System Locale

and ticked the box that says 'Beta: Use Unicode UTF-8 for worldwide language support'. Following a restart, my encoding had changed to 'cp65001' which I believe is equivalent to UTF8 and I was able to create my Lambda Function.

[awood45]

Glad you were able to resolve this! FWIW I've seen this periodically with Python builds especially, pip seems to be very sensitive to encoding issues. Unfortunately, nothing we can do on our end about that.

[mountHouli]

Summary

When I sam build I get the bug, but when I pip install -r requirements.txt I don't get the bug.

Thus, it seems to me to be a SAM CLI issue.

Details

Environment

OS: MacOS Catalina 10.15.7 python: 3.9.7 (installed via pyenv) pip: 21.2.3 (paired with python 3.9.7 installed via pyenv) sam cli: 1.33.0

My requirements.txt file:

atomicwrites==1.4.0; python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.4.0"
attrs==21.2.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
colorama==0.4.4; python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.5.0"
docopt==0.6.2
iniconfig==1.1.1; python_version >= "3.6"
packaging==21.2; python_version >= "3.6"
pluggy==1.0.0; python_version >= "3.6"
py==1.11.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
pyparsing==2.4.7; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6"
pytest-watch==4.2.0
pytest==6.2.5; python_version >= "3.6"
python-dateutil==2.8.2; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.3.0")
python-util==1.2.1
six==1.16.0; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.3.0"
toml==0.10.2; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6"
watchdog==2.1.6; python_version >= "3.6"

Troubleshooting Steps

1. Attempt SAM build

sam build

Building codeuri: /Users/REDACTED/py-root runtime: python3.9 metadata: {} architecture: x86_64 functions: ['VroHtnDataProcessor']
Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {docopt==0.6.2(sdist), pytest-watch==4.2.0(sdist)}

2. Do the pip install

mkdir pip-install-test/
cp requirements.txt pip-install-test/
cd pip-install-test/
pyenv local 3.9.7
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Produces output (which looks ok to me, but I'm not the most advanced python programmer)

Ignoring atomicwrites: markers 'python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.4.0"' don't match your environment
Ignoring colorama: markers 'python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.5.0"' don't match your environment
Collecting attrs==21.2.0
  Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting docopt==0.6.2
  Using cached docopt-0.6.2.tar.gz (25 kB)
Collecting iniconfig==1.1.1
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging==21.2
  Using cached packaging-21.2-py3-none-any.whl (40 kB)
Collecting pluggy==1.0.0
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting py==1.11.0
  Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting pyparsing==2.4.7
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pytest-watch==4.2.0
  Using cached pytest-watch-4.2.0.tar.gz (16 kB)
Collecting pytest==6.2.5
  Using cached pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting python-dateutil==2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting python-util==1.2.1
  Using cached python_util-1.2.1-py3-none-any.whl (11 kB)
Collecting six==1.16.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting toml==0.10.2
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting watchdog==2.1.6
  Using cached watchdog-2.1.6-cp39-cp39-macosx_10_9_x86_64.whl (85 kB)
Collecting colorama>=0.3.3
  Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Using legacy 'setup.py install' for docopt, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pytest-watch, since package 'wheel' is not installed.
Installing collected packages: pyparsing, toml, py, pluggy, packaging, iniconfig, attrs, watchdog, six, pytest, docopt, colorama, python-util, python-dateutil, pytest-watch
    Running setup.py install for docopt ... done
    Running setup.py install for pytest-watch ... done
Successfully installed attrs-21.2.0 colorama-0.4.4 docopt-0.6.2 iniconfig-1.1.1 packaging-21.2 pluggy-1.0.0 py-1.11.0 pyparsing-2.4.7 pytest-6.2.5 pytest-watch-4.2.0 python-dateutil-2.8.2 python-util-1.2.1 six-1.16.0 toml-0.10.2 watchdog-2.1.6
WARNING: You are using pip version 21.2.3; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/REDACTED/venv/bin/python3 -m pip install --upgrade pip' command.

[virajkanwade]

I think I was able to get little more insight into the issue.

MacOS + pyenv

❯ head -n 1 `which sam`
#!/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/bin/python3.8

so the expectation would be following to work

/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/bin/pip3.8 install wheel

After some digging around did this: edit /usr/local/Cellar/aws-sam-cli/1.37.0/site-packages/aws_lambda_builders/workflows/python_pip/packager.py line 658

before or after invoke_pip = [self.python_exe, "-c", exec_string] added print(self.python_exe)

to my surprise it was pointing to /Users/XYZ/.pyenv/shims/python3.8

ran /Users/XYZ/.pyenv/shims/pip3.8 install wheel

and then sam build and it worked.

Please check and confirm here if this works for you too.

[AbdulBasitKhaleeq]

same issue for jsonschema.

Build Failed Error: PythonPipBuilder:ResolveDependencies - {pyrsistent==0.18.1(wheel)}

[croosen]

Same issue with autobahn (although this can happen with any dependency):

Error: PythonPipBuilder:ResolveDependencies - {autobahn==22.3.2(sdist)}

In my case the used version was not compatible (22.3.2). I updated my requirements.txt with:

autobahn==21.2.1

and then run:

pip3 install -r requirements.txt

I found the working version after some searching in the repo of the main package using this dependency but this worked for me.

[maruf897]

@

I solved this using sam build --use-container, however this works only when template.yml file is present. Using the graphical form does not allow to insert --use-container and it fails, so I do not count this as a solution, but workaround instead... // Also since I am unable to build app locally in my IDE, I am unable to run/debug it, so it makes no sense to use this workaround.

how do you test it locally after building with container?

[Leslie-Wong-H]

Just add some extra insights here.

sam project structure as follows: |- template.yaml |- samconfig.toml HelloWorld --- |- requirements.txt --- |- mymodule ------- |- setup.py ------- |- setup.cfg ------- |- src ------------|- code --------------- |- __init__.py --------------- |- code.py

I am using a local module and declare it as "-e relative/path" at requirements.txt.

The problem is that the relative path declared is very tricky.

When sam build --use-container from the project root, it should be -e mymodule/. Otherwise it would cause PythonPipBuilder:ResolveDependencies, yet -e mymodule/ wouldn't work the same when sam build from the project root.

[armando-herastang]

This fixed the ResolveDependencies error for me pip install wheel

This definitely worked for me. But, could someone explain why is this needed? It would be great if it worked out of the box

[KIVagant]

My set of tricks to make it work on Apple M1 (MacOS 11.6 Big Sur)

brew remove aws-sam-cli # will be installed below using pip

brew install pyenv
# Also you may need to remove an existing python version.
# I had to remove and reinstall it back later after all other commands to make things work
# brew remove python@3.9
pyenv install 3.7.13   # < 3.7 is currently required by AWS SAM
pyenv global 3.7.13

vim ~/.zshrc || vim ~/.bashrc

### add these lines to the *rc file:

  # >> PYENV
  export PYENV_ROOT="$HOME/.pyenv"
  export PATH="$PYENV_ROOT/bin:$PATH"
  eval "$(pyenv init --path)"
  # << PYENV

#save and close the file

#open a new terminal window

pip install aws-sam-cli wheel

# you may also need this if you have Podman, even if you build locally:
# pip install paramiko

Check versions:

python -V && pip -V && sam --version

# Python 3.7.13
# pip 22.0.4 from /Users/yevhen.hlotov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/pip (python 3.7)
# SAM CLI, version 1.53.0

Build the project with sam build and it should work now. Although, it still cannot build in containers when using Podman due to the separate issue.

[austinrbrown01]

To anybody who sees this in the future, the I was having the same issue as this thread (and like others, I was able to resolve using sam build --use-container. It turns out that I didn't have a virtual environment set up in my project root, which makes sense why this would create dependency issues.

I only figured this out because my deployment package became too large to deploy, and I realized it was bundling a ton of packages I didn't need. This led me to realize I didn't have an virtual env configured.

[LoganB99]

I am still seeing this issue with sam build --use-container. Using podman on macOS (tried PKG, homebrew, and now pyenv. Using DOCKER_HOST. sam local invoke and sam local start-api only work with ---skip-pull-image. sam build without use-container works

Build Failed Running PythonPipBuilder:ResolveDependencies Error: PythonPipBuilder:ResolveDependencies - Could not satisfy the requirement: requests

trying the hello world example

[erikfinnman]

On the surface, my problem seemed to be the same as many others here. But it turns out that my issue was an expired token to our CodeArtifact repository (which we use for all our Python dependencies). This became evident when using the --debug option and seeing that sam build was unable to download any dependencies.

[sergiors]

I have the same

Build Failed
 Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - {pendulum==2.0.5(sdist)}
make: *** [build] Error 1

[VctrEman]

I have the same, but for another package: lightgbm 4.2.0

Final incompatible: {lightgbm==4.1.0(wheel)} Final missing wheels: {lightgbm==4.1.0(sdist)} PythonPipBuilder:ResolveDependencies failed

I had to downgrade my lgbm version to 3.3.5 and it the build worked, testing locally too.