serverless deploy -> Exited with code 1

StayFoolisj commented 2 years ago

Are you certain it's a bug?

[X] Yes, it looks like a bug

Is the issue caused by a plugin?

[X] It is not a plugin issue

Are you using the latest version?

[X] Yes, I'm using the latest version

Is there an existing issue for this?

[X] I have searched existing issues, it hasn't been reported yet

Issue description

Error: docker run --rm -v /Users/bad_ape/Library/Caches/serverless-python-requirements/36d8e2271e2d9e8afb4b103da1d57dc91ab0ccc62422d1337b5a3a91dcc7c2d9_x86_64_slspyc:/var/task:z -v /Users/bad_ape/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 lambci/lambda:build-python3.8 python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache Exited with code 1

Service configuration (serverless.yml) content

service: liqs

frameworkVersion: '2'

provider:
  name: aws
  runtime: python3.8
  lambdaHashingVersion: 20201221

functions:
  hello:
    handler: lambda_function.hello

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    dockerizePip: non-linux

Command name and used flags

serverless deploy

Command output

Serverless: Running "serverless" installed locally (in service node_modules)
Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command requirements
Serverless: Load command requirements:clean
Serverless: Load command requirements:install
Serverless: Load command requirements:cleanCache
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Skipping variables resolution with old resolver (new resolver reported no more variables to resolve)
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Generated requirements from /Users/bad_ape/Downloads/liquidations_v2-8d3d697e-5a0a-4f55-9920-31ee9b80ac90/liqs_pipeline_1/requirements.txt in /Users/bad_ape/Downloads/liquidations_v2-8d3d697e-5a0a-4f55-9920-31ee9b80ac90/liqs_pipeline_1/.serverless/requirements.txt...
Serverless: Installing requirements from /Users/bad_ape/Library/Caches/serverless-python-requirements/36d8e2271e2d9e8afb4b103da1d57dc91ab0ccc62422d1337b5a3a91dcc7c2d9_x86_64_slspyc/requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.8
Serverless: Using download cache directory /Users/bad_ape/Library/Caches/serverless-python-requirements/downloadCacheslspyc
Serverless: Running docker run --rm -v /Users/bad_ape/Library/Caches/serverless-python-requirements/36d8e2271e2d9e8afb4b103da1d57dc91ab0ccc62422d1337b5a3a91dcc7c2d9_x86_64_slspyc\:/var/task\:z -v /Users/bad_ape/Library/Caches/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z -u 0 lambci/lambda\:build-python3.8 python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache...
Serverless: [AWS sts 200 1.37s 0 retries] getCallerIdentity({})

 Error ---------------------------------------------------

  Error: `docker run --rm -v /Users/bad_ape/Library/Caches/serverless-python-requirements/36d8e2271e2d9e8afb4b103da1d57dc91ab0ccc62422d1337b5a3a91dcc7c2d9_x86_64_slspyc:/var/task:z -v /Users/bad_ape/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 lambci/lambda:build-python3.8 python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache` Exited with code 1
      at ChildProcess.<anonymous> (/Users/bad_ape/Downloads/liquidations_v2-8d3d697e-5a0a-4f55-9920-31ee9b80ac90/liqs_pipeline_1/node_modules/child-process-ext/spawn.js:38:8)
      at ChildProcess.emit (node:events:390:28)
      at ChildProcess.emit (node:domain:475:12)
      at maybeClose (node:internal/child_process:1062:16)
      at Socket.<anonymous> (node:internal/child_process:448:11)
      at Socket.emit (node:events:390:28)
      at Socket.emit (node:domain:475:12)
      at Pipe.<anonymous> (node:net:687:12)

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              17.2.0
     Framework Version:         2.72.0 (local)
     Plugin Version:            5.5.3
     SDK Version:               4.3.0
     Components Version:        3.18.1

Environment information

Serverless: Running "serverless" installed locally (in service node_modules)
Framework Core: 2.72.0 (local)
Plugin: 5.5.3
SDK: 4.3.0
Components: 3.18.1

MacOS Monterey 12.0.1
MB Pro 2021
Chip: Apple M1 Pro

medikoo commented 2 years ago

Issue looks as specific to serverless-python-requirements plugin, moving it there

ftruzzi commented 2 years ago

Seeing this issue on Gitlab CI. ~~Possible solution: downgrade node to v16.~~ not sure, see below

vivianedias-tw commented 2 years ago

hey @medikoo I'm facing the same issue here. Where did you move the issue?

medikoo commented 2 years ago

Where did you move the issue?

Here

patalwell commented 2 years ago

I tried Node V16, still get a the same error on my mac.

 Error ---------------------------------------------------

  Error: `docker run --rm -v /Users/patalwell/Library/Caches/serverless-python-requirements/f4b4cffcac01d03e573bc9fb87d6d9e96378ee0b2e232c53d182e16ca5364748_x86_64_slspyc:/var/task:z -v /Users/patalwell/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -e AWS_CA_BUNDLE=/Users/patalwell/Certificates/ca_bundle.pem -u 0 lambci/lambda:build-python3.7 /bin/sh -c python3.7 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && find /var/task -name \*.so -exec strip \{\} \;` Exited with code 1
      at ChildProcess.<anonymous> (/Users/patalwell/Development/cloudshield-reports-aggregator/cloudshield-reports-aggregator/node_modules/child-process-ext/spawn.js:38:8)
      at ChildProcess.emit (node:events:390:28)
      at ChildProcess.emit (node:domain:475:12)
      at maybeClose (node:internal/child_process:1064:16)
      at Socket.<anonymous> (node:internal/child_process:450:11)
      at Socket.emit (node:events:390:28)
      at Socket.emit (node:domain:475:12)
      at Pipe.<anonymous> (node:net:687:12)

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              16.13.2
     Framework Version:         2.71.0
     Plugin Version:            5.5.3
     SDK Version:               4.3.0
     Components Version:        3.18.1

ftruzzi commented 2 years ago

@patalwell uh sorry, the issue must be somewhere else then. Now that I looked again I'm seeing failures with node 16 as well.

All I know is that on my CI it fails using nikolaik/python-nodejs:latest (which is node 16 anyway) as docker base image and it works with nikolaik/python-nodejs:python3.8-nodejs16. So maybe the Python version, or something else?

vivianedias-tw commented 2 years ago

In order to use those two versions you mentioned should I set this docker image in the serverless config? @ftruzzi

ftruzzi commented 2 years ago

@vivianedias-tw no, I was just saying that, given the same serverless config and packages, it fails with serverless running under the first docker image and it works under the second one

patalwell commented 2 years ago

Tried with python 3.7 and python3.8, still same issue.

provider:
  name: aws
  runtime: python3.8 | python3.7

  #  lambci/lambda:build-python3.8

Defaults to container for lambda builds and runtime selected.

biximilien commented 2 years ago

I'm experiencing the same issue, I think it is due to this docker run command:

docker run --rm -v /home/biximilien/Projects/ACN/be-lambdas/services/tag/.serverless/requirements:/var/task:z -v /home/biximilien/.cache/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z sls-py-reqs-custom /bin/sh -c chown -R 0\:0 /var/useDownloadCache && python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 1000\:1000 /var/task && cp /usr/lib64/mysql57/libmysqlclient.so.1020 /var/task/ && chown -R 1000\:1000 /var/useDownloadCache && find /var/task -name \*.so -exec strip \{\} \;` Exited with code 1

When running the first part:

docker run --rm -v /home/biximilien/Projects/ACN/be-lambdas/services/tag/.serverless/requirements:/var/task:z -v /home/biximilien/.cache/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z sls-py-reqs-custom /bin/sh -c chown -R 0\:0 /var/useDownloadCache 
chown: missing operand

Adding quotes around the bin/sh'd command string fixes it.

docker run --rm -v /home/biximilien/Projects/ACN/be-lambdas/services/tag/.serverless/requirements:/var/task:z -v /home/biximilien/.cache/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z sls-py-reqs-custom /bin/sh -c "chown -R 0\:0 /var/useDownloadCache && python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 1000\:1000 /var/task && cp /usr/lib64/mysql57/libmysqlclient.so.1020 /var/task/ && chown -R 1000\:1000 /var/useDownloadCache && find /var/task -name \*.so -exec strip \{\} \;"

I'll try to submit a pull request

biximilien commented 2 years ago

I'll try to submit a pull request ~~See #664~~

agneeshdg commented 2 years ago

I'm facing almost the same error in a Python 3.9 runtime, but with a different error code of 125 on my Mac: Error: docker run --rm -v /Users/adasgup/apps/myproject/.serverless/requirements:/var/task:z -v /Users/adasgup/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 lambci/lambda:build-python3.9 python3.9 -m pip install --no-cache-dir --find-links reqs/wheels -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache Exited with code 125

StayFoolisj commented 2 years ago

I'm facing almost the same error in a Python 3.9 runtime, but with a different error code of 125 on my Mac: Error: docker run --rm -v /Users/adasgup/apps/myproject/.serverless/requirements:/var/task:z -v /Users/adasgup/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 lambci/lambda:build-python3.9 python3.9 -m pip install --no-cache-dir --find-links reqs/wheels -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache Exited with code 125

I believe this is the error you get when Docker is not running?

pgrzesik commented 2 years ago

Hello everyone, thanks a lot for the reports. I've tried to reproduce the issue but with no luck - it works fine for me with the following config + requirements.

service: python-docker-bug

provider:
  name: aws
  runtime: python3.8
  lambdaHashingVersion: 20201221

functions:
  hello:
    handler: lambda_function.hello

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    dockerizePip: non-linux

requests

I'm using Node v16.13.0 and I'm on a Mac (Big Sur, M1 chip). Could anyone provide a complete reproducible example, ideally with requirements/pipfile/pyproject.toml ?

biximilien commented 2 years ago

I did some more testing this morning and confirmed the fix I suggested was in fact not working.

The issue was that I confused the serverless error description with the command being ran, the description had the command not properly quoted, but the command being ran was properly quoted.

In my case, the cause of the error was a file not found in this setting:

dockerExtraFiles:
      - /usr/lib64/mysql/libmysqlclient.so.21

Resolving this path fixed my problem.

agneeshdg commented 2 years ago

I followed the instructions on the tutorial to start and deploy a new project, and am now getting the exit code 1 error. https://www.serverless.com/blog/serverless-python-packaging/ My requirements are just: numpy==1.22.1

For the tutorial, I am running it in a Python 3.6 runtime on a Mac with Intel Core i7 processor.

pgrzesik commented 2 years ago

Can you share a bit more @agneeshdg ? Do you have docker installed locally? I've tried to reproduce with numpy but it also works correctly

agneeshdg commented 2 years ago

Hi @pgrzesik I do have Docker installed and running locally. Is there any additional Docker configuration I need to do?

patalwell commented 2 years ago

@pgrzesik let me try to use your config params. I'm not sure non-linux would work with serverless since lambda inherently uses a linux OS. Glad to see Docker community has support for M1 chip though! That was a pain point for me in the past. Did you also happen to run the lambda?

patalwell commented 2 years ago

@pgrzesik tested your params, same issues. What does your package.json look like?

pgrzesik commented 2 years ago

I'm not sure non-linux would work with serverless since lambda inherently uses a linux OS.

non-linux means that it will use Docker for packages installation/building instead of your native OS for that exact reason. I didn't deploy the Lambda afterward but the building process went just fine. As for my package.json - it only has servelress-python-requirements as a dev dependency.

@patalwell - could you upload a small repro example that I could run as-is on my side? Maybe there's something niche in our setups that makes the difference?

pgrzesik commented 2 years ago

Hi @pgrzesik I do have Docker installed and running locally. Is there any additional Docker configuration I need to do?

Nope, that should be enough - could you run it with --verbose flag and/or SLS_DEBUG=1 env var for getting more context for the error you're getting?

mgermy commented 2 years ago

I'm having exactly the same problem, but instead of Exited with code 1 it says code 2. Not sure what to do 😕

pgrzesik commented 2 years ago

Hello @mgermy 👋 Would you be able to provide more context around your specific case? Maybe you could provide a small reproducible example that I can run on my side?

mgermy commented 2 years ago

@pgrzesik I created an example so you could run the same on your side. I only omitted my AWS role and profile. My node version is v16.13.2.

Below you'll find a screenshot of the error and the files used to run it.

aws-python-project.zip

mgermy commented 2 years ago

I tried running the docker run command and it gives me the following error:

Maybe it's something related to some permission problems?

patalwell commented 2 years ago

Looks like you have a symlink or dependecy issue

I tried running the docker run command and it gives me the following error:

Maybe it's something related to some permission problems?

You'e trying to run Python 3.9 with Docker, but I'm seeing stack traces for Python 3.8. I'd make sure the version is what you expect and the permissions for the Docker mounts are tangible ; e.g docker is able to mount to your local disk.

pgrzesik commented 2 years ago

Thanks a lot @mgermy - I've tried to reproduce your specific issue and it seems like the problem is the python3.9 runtime. When packaging with docker, it tries to do it with lambci/lambda:build-python3.9 image that is not available - the latest available is for python3.8. I think the options here would be to

Check before trying to run packaging if the runtime is supported with docker, if not error out
Bring support for python3.9 runtime with docker

@patalwell Did you manage to resolve the problem on your side?

mgermy commented 2 years ago

@pgrzesik thanks for taking for taking a look at it for me.

After a few hours trying to figure it out, I found out that the problem was with the installation of serverless on my computer. I suggest for anyone who is having the same problem to check with a simple ls -la if the folder where you're running serverless has enough permission to run. For example, in my folder I had the .serverless folder with the ownership for root, not my user. Once I changed the ownership to my user, serverless was able to run without problems

pgrzesik commented 2 years ago

Thanks a lot for sharing the solution to your problem @mgermy 👍

Sillium commented 2 years ago

FWIW: I have the same problem (serverless 2.72.0 and 3.1.1) and ownership of the .serverless folder is correctly set to my user, not root.

pgrzesik commented 2 years ago

@Sillium Are you able to share a repro example of your specific setup that is causing you trouble?

Eggwise commented 2 years ago

Had the same error.

But i discovered i had a unrecognized dependency defined in the requirements.txt webflow had to be webflowpy.

Check your requirements.txt if its correct

reznok commented 2 years ago

Running on windows. Noticed the requirements.txt in the cache (C:\Users\<USER>\AppData\Local\UnitedIncome\serverless-python-requirements\Cache\4672ad8e59652f4826562bf60e12f81a52440989b3b56d960144485299041ce0_x86_64_slspyc\requirements.txt) was mangled, causing errors. Appears to be an issue with copying it over to the cache.

requirements.txt before copy (in my project directory):

click==8.0.3
colorama==0.4.4
Flask==2.0.3
itsdangerous==2.0.1
Jinja2==3.0.3
MarkupSafe==2.0.1
Werkzeug==2.0.3

requirements.txt after copy in the cache:

 F l a s k = = 2 . 0 . 3 

 J i n j a 2 = = 3 . 0 . 3 

 M a r k u p S a f e = = 2 . 0 . 1 

 W e r k z e u g = = 2 . 0 . 3 
 c o l o r a m a = = 0 . 4 . 4 

 i t s d a n g e r o u s = = 2 . 0 . 1 

��c l i c k = = 8 . 0 . 3

This is causing similar Exit Code 1 errors as previous people have reported.

A short-term work around is to use pipenv.

Sillium commented 2 years ago

@Sillium Are you able to share a repro example of your specific setup that is causing you trouble?

Hello @pgrzesik, sorry for the delayed response. This repo (branch "serverless-python-requirements-bug") shows the error when deploying: https://github.com/Sillium/heutejournal/tree/serverless-python-requirements-bug

zhemaituk commented 2 years ago

Is it possible to print out the docker run ... command output during sls deploy somehow? SLS_DEBUG=1 does not do it.

dtosato commented 2 years ago

serverless-python-requirements@5.3.1

$ sls deploy --stage dev
Running "serverless" from node_modules
Deploying app to stage dev (us-west-2)
× Stack app failed to deploy (3s)
Environment: linux, node 14.19.0, framework 3.3.0 (local) 3.3.0v (global), plugin 6.1.0, SDK 4.3.1
Credentials: Local, environment variables
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues
Error:
Error: `python3.9 -m pip install -t /root/.cache/serverless-python-requirements/d4b5439e56528d78e43e64ec97f05418f79486d582f391ecec1f13247c4af4fd_x86_64_slspyc -r /root/.cache/serverless-python-requirements/d4b5439e56528d78e43e64ec97f05418f79486d582f391ecec1f13247c4af4fd_x86_64_slspyc/requirements.txt --cache-dir /root/.cache/serverless-python-requirements/downloadCacheslspyc` Exited with code 1
    at ChildProcess.<anonymous> (/builds/app/node_modules/child-process-ext/spawn.js:38:8)
    at ChildProcess.emit (events.js:400:28)
    at ChildProcess.emit (domain.js:475:12)
    at maybeClose (internal/child_process.js:1058:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)

pgrzesik commented 2 years ago

Hey @Sillium - I've managed to reproduce and in your case the issues is that regular pip install of the provided requirements is failing - do you observe pip install working correctly on your machine for your set of credenitals?

@dtosato - could you share anything more about your setup? It's hard to reason about only error message in this case.

@zhemaituk I believe that's what we need to expose as most of there errors are e.g. some internal pip errors that are just swallowed. We'd be happy to accept a contribution that exposes these error logs both for docker and non-docker installations 🙇

Sillium commented 2 years ago

Hey @Sillium - I've managed to reproduce and in your case the issues is that regular pip install of the provided requirements is failing - do you observe pip install working correctly on your machine for your set of credenitals?

Thanks @pgrzesik, that was the reason in my case! When I fixed my requirements.txt to work with pip install again, the deployment went through. Sorry for wasting your time on an error like this.

dtosato commented 2 years ago

Hi @pgrzesik, I have this problem on both my dev machine and on the gitlab ci runner. The runner uses the nikolaik/python-nodejs:python3.9-nodejs14 doker image with the following setup:

  - npm i -g npm@latest
  - npm config set prefix /usr/local
  - npm install -g serverless@latest
  - pip install --upgrade pip
  - pip install poetry
  - poetry config virtualenvs.create false

Down here the complete log.

Running with gitlab-runner 14.8.0~beta.44.g57df0d52 (57df0d52)
  on blue-5.shared.runners-manager.gitlab.com/default -AzERasQ
Resolving secrets
00:00
Preparing the "docker+machine" executor
00:38
Using Docker executor with image nikolaik/python-nodejs:python3.9-nodejs14 ...
Pulling docker image nikolaik/python-nodejs:python3.9-nodejs14 ...
Using docker image sha256:730d31d2ec65606d61a24de88dfd38de2f2e5d3db3e4b86b55948c04d62cc4be for nikolaik/python-nodejs:python3.9-nodejs14 with digest nikolaik/python-nodejs@sha256:78669f9572203fa093c8bea3651a803bd223638549aa9576b0187d89b2bdd90a ...
Preparing environment
00:02
Running on runner--azerasq-project-25827046-concurrent-0 via runner-azerasq-shared-1645607364-7c17864b...
Getting source from Git repository
00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/x/mash/app/.git/
Created fresh repository.
Checking out d885d712 as master...
Skipping Git submodules setup
Restoring cache
00:04
Checking cache for master...
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/25827046/master 
Successfully extracted cache
Executing "step_script" stage of the job script
01:27
Using docker image sha256:730d31d2ec65606d61a24de88dfd38de2f2e5d3db3e4b86b55948c04d62cc4be for nikolaik/python-nodejs:python3.9-nodejs14 with digest nikolaik/python-nodejs@sha256:78669f9572203fa093c8bea3651a803bd223638549aa9576b0187d89b2bdd90a ...
$ npm i -g npm@latest
/usr/bin/npm -> /usr/lib/node_modules/npm/bin/npm-cli.js
/usr/bin/npx -> /usr/lib/node_modules/npm/bin/npx-cli.js
+ npm@8.5.1
added 71 packages from 18 contributors, removed 291 packages and updated 146 packages in 9.001s
$ npm config set prefix /usr/local
$ npm install -g serverless@latest
npm WARN deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm WARN deprecated querystring@0.2.1: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated uuid@3.3.2: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated superagent@3.8.3: Please upgrade to v7.0.2+ of superagent.  We have fixed numerous issues with streams, form-data, attach(), filesystem errors not bubbling up (ENOENT on attach()), and all tests are now passing.  See the releases tab for more information at <https://github.com/visionmedia/superagent/releases>.
added 389 packages, and audited 390 packages in 21s
52 packages are looking for funding
  run `npm fund` for details
found 0 vulnerabilities
$ npm i
up to date, audited 420 packages in 1s
54 packages are looking for funding
  run `npm fund` for details
found 0 vulnerabilities
$ pip install --upgrade pip
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (22.0.2)
Collecting pip
  Downloading pip-22.0.3-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 26.4 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.0.2
    Uninstalling pip-22.0.2:
      Successfully uninstalled pip-22.0.2
Successfully installed pip-22.0.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ pip install poetry
Collecting poetry
  Downloading poetry-1.1.13-py2.py3-none-any.whl (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 KB 6.0 MB/s eta 0:00:00
Collecting tomlkit<1.0.0,>=0.7.0
  Downloading tomlkit-0.10.0-py3-none-any.whl (33 kB)
Requirement already satisfied: virtualenv<21.0.0,>=20.0.26 in /usr/local/lib/python3.9/site-packages (from poetry) (20.13.0)
Collecting shellingham<2.0,>=1.1
  Using cached shellingham-1.4.0-py2.py3-none-any.whl (9.4 kB)
Collecting html5lib<2.0,>=1.0
  Using cached html5lib-1.1-py2.py3-none-any.whl (112 kB)
Collecting cachy<0.4.0,>=0.3.0
  Using cached cachy-0.3.0-py2.py3-none-any.whl (20 kB)
Collecting cachecontrol[filecache]<0.13.0,>=0.12.9
  Using cached CacheControl-0.12.10-py2.py3-none-any.whl (20 kB)
Collecting crashtest<0.4.0,>=0.3.0
  Using cached crashtest-0.3.1-py3-none-any.whl (7.0 kB)
Collecting poetry-core<1.1.0,>=1.0.7
  Using cached poetry_core-1.0.7-py2.py3-none-any.whl (424 kB)
Collecting pexpect<5.0.0,>=4.7.0
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting packaging<21.0,>=20.4
  Using cached packaging-20.9-py2.py3-none-any.whl (40 kB)
Collecting requests-toolbelt<0.10.0,>=0.9.1
  Using cached requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
Collecting pkginfo<2.0,>=1.4
  Using cached pkginfo-1.8.2-py2.py3-none-any.whl (26 kB)
Collecting cleo<0.9.0,>=0.8.1
  Using cached cleo-0.8.1-py2.py3-none-any.whl (21 kB)
Collecting requests<3.0,>=2.18
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting keyring>=21.2.0
  Downloading keyring-23.5.0-py3-none-any.whl (33 kB)
Collecting clikit<0.7.0,>=0.6.2
  Using cached clikit-0.6.2-py2.py3-none-any.whl (91 kB)
Collecting msgpack>=0.5.2
  Using cached msgpack-1.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB)
Collecting lockfile>=0.9
  Using cached lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Collecting pastel<0.3.0,>=0.2.0
  Using cached pastel-0.2.1-py2.py3-none-any.whl (6.0 kB)
Collecting pylev<2.0,>=1.3
  Using cached pylev-1.4.0-py2.py3-none-any.whl (6.1 kB)
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.9/site-packages (from html5lib<2.0,>=1.0->poetry) (1.16.0)
Collecting importlib-metadata>=3.6
  Downloading importlib_metadata-4.11.1-py3-none-any.whl (17 kB)
Collecting SecretStorage>=3.2
  Using cached SecretStorage-3.3.1-py3-none-any.whl (15 kB)
Collecting jeepney>=0.4.2
  Using cached jeepney-0.7.1-py3-none-any.whl (54 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-3.0.7-py3-none-any.whl (98 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry) (2021.10.8)
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry) (0.3.4)
Requirement already satisfied: platformdirs<3,>=2 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry) (2.4.1)
Requirement already satisfied: filelock<4,>=3.2 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry) (3.4.2)
Collecting zipp>=0.5
  Downloading zipp-3.7.0-py3-none-any.whl (5.3 kB)
Collecting cryptography>=2.0
  Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting cffi>=1.12
  Using cached cffi-1.15.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (444 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: webencodings, pylev, ptyprocess, pkginfo, msgpack, lockfile, zipp, urllib3, tomlkit, shellingham, pyparsing, pycparser, poetry-core, pexpect, pastel, jeepney, idna, html5lib, crashtest, charset-normalizer, cachy, requests, packaging, importlib-metadata, clikit, cffi, requests-toolbelt, cryptography, cleo, cachecontrol, SecretStorage, keyring, poetry
Successfully installed SecretStorage-3.3.1 cachecontrol-0.12.10 cachy-0.3.0 cffi-1.15.0 charset-normalizer-2.0.12 cleo-0.8.1 clikit-0.6.2 crashtest-0.3.1 cryptography-36.0.1 html5lib-1.1 idna-3.3 importlib-metadata-4.11.1 jeepney-0.7.1 keyring-23.5.0 lockfile-0.12.2 msgpack-1.0.3 packaging-20.9 pastel-0.2.1 pexpect-4.8.0 pkginfo-1.8.2 poetry-1.1.13 poetry-core-1.0.7 ptyprocess-0.7.0 pycparser-2.21 pylev-1.4.0 pyparsing-3.0.7 requests-2.27.1 requests-toolbelt-0.9.1 shellingham-1.4.0 tomlkit-0.10.0 urllib3-1.26.8 webencodings-0.5.1 zipp-3.7.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ poetry config virtualenvs.create false
$ poetry config repositories.packagr https://api.packagr.app/xxxxxx/
$ poetry config http-basic.packagr $POETRY_HTTP_BASIC_PYPI_USERNAME $POETRY_HTTP_BASIC_PYPI_PASSWORD
No suitable keyring backends were found
Using a plaintext file to store and retrieve credentials
$ poetry install --no-dev
Skipping virtualenv creation, as specified in config file.
Installing dependencies from lock file
Package operations: 14 installs, 0 updates, 2 removals
  • Removing packaging (21.3)
  • Removing pyparsing (3.0.7)
  • Installing jmespath (0.10.0)
  • Installing python-dateutil (2.8.2)
  • Installing botocore (1.24.5)
  • Installing ciso8601 (2.2.0)
  • Installing numpy (1.22.2)
  • Installing pytz (2021.3)
  • Installing rx (3.2.0)
  • Installing s3transfer (0.5.1)
  • Installing boto3 (1.21.5)
  • Installing influxdb-client (1.26.0)
  • Installing pymongo (4.0.1)
  • Installing pandas (1.4.1)
  • Installing mash (4.7.0)
  • Installing python-dotenv (0.19.2)
$ sls deploy --stage dev
Running "serverless" from node_modules
Deploying app to stage dev (us-west-2)
× Stack app-dev failed to deploy (3s)
Environment: linux, node 14.19.0, framework 3.3.0 (local) 3.3.0v (global), plugin 6.1.0, SDK 4.3.1
Credentials: Local, environment variables
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues
Error:
Error: `python3.9 -m pip install -t /root/.cache/serverless-python-requirements/d4b5439e56528d78e43e64ec97f05418f79486d582f391ecec1f13247c4af4fd_x86_64_slspyc -r /root/.cache/serverless-python-requirements/d4b5439e56528d78e43e64ec97f05418f79486d582f391ecec1f13247c4af4fd_x86_64_slspyc/requirements.txt --cache-dir /root/.cache/serverless-python-requirements/downloadCacheslspyc` Exited with code 1
    at ChildProcess.<anonymous> (/builds/x/mash/app/node_modules/child-process-ext/spawn.js:38:8)
    at ChildProcess.emit (events.js:400:28)
    at ChildProcess.emit (domain.js:475:12)
    at maybeClose (internal/child_process.js:1058:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1

ychouinardgc commented 2 years ago

Instead of just showing Exited with code 1, Is there a way to see the output of the pip install command?

zhemaituk commented 2 years ago

@pgrzesik done, printing stdout/stderr on failure, quite straightforward and maybe naive but it works and prints details: https://github.com/serverless/serverless-python-requirements/compare/master...zhemaituk:patch-1

dtosato commented 2 years ago

@pgrzesik now it is working to me, I have just updated my npm dependencies.

pgrzesik commented 2 years ago

@zhemaituk - I believe this approach would be just fine for debugging, especially that it's on info level. Would you like to submit a PR with such change? I think it would tremendously help a lot of people that have trouble in this thread

stevetran-96 commented 2 years ago

@reznok I believe I'm having the exact same issue as you. It seems to be an issue when generating/copying the requirements.txt. I noticed that after the step: Generated requirements from C:\Users\<USER>\Desktop\Root\requirements.txt in C:\Users\<USER>\Desktop\Root\.serverless\requirements.txt The requirements.txt in .serverless folder was also broken. Have you found a solution for this?

Update: I found this after a bit of searching:

This is an open issue as on date 10th March 2022 The serverless plugin rewrites the file into .serverless directory & it assumes UTF8 encoding when it reads the file. The problem occurs as serverless attempts to read a file with another encoding & then dump it into .serverless folder. https://github.com/serverless/serverless-python-requirements/issues/442

takeda commented 2 years ago

I see similar issue, but when invoking python3.9 -m pip help install

00:00:09 Environment: linux, node 14.19.0, framework 3.7.8 (local), plugin 6.1.5, SDK 4.3.2
00:00:09 Credentials: Local, environment variables
00:00:09 Docs:        docs.serverless.com
00:00:09 Support:     forum.serverless.com
00:00:09 Bugs:        github.com/serverless/serverless/issues
00:00:09 
00:00:09 Error:
00:00:09 Error: `python3.9 -m pip help install` Exited with code 1
00:00:09     at ChildProcess.<anonymous> (/nix/store/gna6q42v03acmxnxqkrbvvj0krba24ry-amiman-0.1.0/node_modules/child-process-ext/spawn.js:38:8)
00:00:09     at ChildProcess.emit (events.js:400:28)
00:00:09     at ChildProcess.emit (domain.js:475:12)
00:00:09     at maybeClose (internal/child_process.js:1058:16)
00:00:09     at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)

Interestingly everything works perfectly fine when using python3.8 (yes, I cleaned cache between runs)

HansonVPHAM commented 2 years ago

I got the same issue but when I check the requirements.txt file and I see one version of my library was not correct at all. Please make sure requirements.txt is correct.

jaren82 commented 2 years ago

I'm facing almost the same error in a Python 3.9 runtime, but with a different error code of 125 on my Mac: Error: docker run --rm -v /Users/adasgup/apps/myproject/.serverless/requirements:/var/task:z -v /Users/adasgup/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 lambci/lambda:build-python3.9 python3.9 -m pip install --no-cache-dir --find-links reqs/wheels -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache Exited with code 125

Hi I had the same problem So I found a solution and tried it And I succeeded You try delete unused docker images.

docker volume rm $(docker volume ls -qf dangling=true)

docker rmi $(docker images | grep "^<none>" | awk "{print $3}")
docker images -a | sed '1 d' | awk '{print $3}' | xargs -L1 docker rmi -f

lorisleitner commented 2 years ago

I got the error when pip freeze > requirements.txt was written in UTF-16 encoding. Changed the encoding to UTF-8 and it's working now.

takeda commented 2 years ago

I think it's obvious that while https://github.com/serverless/serverless-python-requirements/pull/679 added extra logging, there's still more places where logs need to be added as different people have different reasons why pip "Exited with code 1".

serverless / serverless-python-requirements