search

youyo / aws-cdk-github-actions

MIT License
207 stars 101 forks source link

Packaging not working when deploying lambda function packaged using docker #50

Open trobert2 opened 2 years ago

trobert2 commented 2 years ago

When deploying a PythonFunction using aws_lambda_python, the packaging takes place in a docker container. The packaging of the function, including automatic installation of the requirements.txt dependencies is done inside the container.

Locally the execution happens without any issues. Using the github actions this fails workflow:

on:
  push:
    branches: [ main ]

jobs:
  aws_cdk:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: cdk diff
        uses: youyo/aws-cdk-github-actions@v2
        with:
          cdk_subcommand: 'diff'
          cdk_version: '2.9.0'
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_DEFAULT_REGION: 'eu-central-1'

Error AFTER installation from requirements.txt has taken place, states that requirements.txt is not present:

---> Running in 16c650c9a013
Collecting pipenv
  Downloading pipenv-2022.1.8-py2.py3-none-any.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 55.3 MB/s eta 0:00:00
Collecting poetry
  Downloading poetry-1.1.12-py2.py3-none-any.whl (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.0/175.0 KB 34.5 MB/s eta 0:00:00
Requirement already satisfied: setuptools>=36.2.1 in /var/lang/lib/python3.9/site-packages (from pipenv) (58.1.0)
Requirement already satisfied: pip>=18.0 in /var/lang/lib/python3.9/site-packages (from pipenv) (22.0.2)
Collecting virtualenv-clone>=0.2.5
  Downloading virtualenv_clone-0.5.7-py3-none-any.whl (6.6 kB)
Collecting virtualenv
  Downloading virtualenv-20.13.0-py2.py3-none-any.whl (6.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 74.9 MB/s eta 0:00:00
Collecting certifi
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.2/149.2 KB 32.7 MB/s eta 0:00:00
Collecting pexpect<5.0.0,>=4.7.0
  Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.0/59.0 KB 15.8 MB/s eta 0:00:00
Collecting poetry-core<1.1.0,>=1.0.7
  Downloading poetry_core-1.0.7-py2.py3-none-any.whl (424 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 424.8/424.8 KB 50.3 MB/s eta 0:00:00
Collecting cleo<0.9.0,>=0.8.1
  Downloading cleo-0.8.1-py2.py3-none-any.whl (21 kB)
Collecting cachecontrol[filecache]<0.13.0,>=0.12.9
  Downloading CacheControl-0.12.10-py2.py3-none-any.whl (20 kB)
Collecting cachy<0.4.0,>=0.3.0
  Downloading cachy-0.3.0-py2.py3-none-any.whl (20 kB)
Collecting html5lib<2.0,>=1.0
  Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 112.2/112.2 KB 15.7 MB/s eta 0:00:00
Collecting pkginfo<2.0,>=1.4
  Downloading pkginfo-1.8.2-py2.py3-none-any.whl (26 kB)
Collecting requests<3.0,>=2.18
  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.1/63.1 KB 15.1 MB/s eta 0:00:00
Collecting shellingham<2.0,>=1.1
  Downloading shellingham-1.4.0-py2.py3-none-any.whl (9.4 kB)
Collecting requests-toolbelt<0.10.0,>=0.9.1
  Downloading requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.3/54.3 KB 14.8 MB/s eta 0:00:00
Collecting clikit<0.7.0,>=0.6.2
  Downloading clikit-0.6.2-py2.py3-none-any.whl (91 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 91.8/91.8 KB 25.2 MB/s eta 0:00:00
Collecting tomlkit<1.0.0,>=0.7.0
  Downloading tomlkit-0.8.0-py3-none-any.whl (33 kB)
Collecting keyring<22.0.0,>=21.2.0
  Downloading keyring-21.8.0-py3-none-any.whl (32 kB)
Collecting crashtest<0.4.0,>=0.3.0
  Downloading crashtest-0.3.1-py3-none-any.whl (7.0 kB)
Collecting packaging<21.0,>=20.4
  Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 KB 11.6 MB/s eta 0:00:00
Collecting msgpack>=0.5.2
  Downloading msgpack-1.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 322.2/322.2 KB 49.6 MB/s eta 0:00:00
Collecting lockfile>=0.9
  Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Collecting pylev<2.0,>=1.3
  Downloading pylev-1.4.0-py2.py3-none-any.whl (6.1 kB)
Collecting pastel<0.3.0,>=0.2.0
  Downloading pastel-0.2.1-py2.py3-none-any.whl (6.0 kB)
Collecting six>=1.9
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting webencodings
  Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting SecretStorage>=3.2
  Downloading SecretStorage-3.3.1-py3-none-any.whl (15 kB)
Collecting jeepney>=0.4.2
  Downloading jeepney-0.7.1-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.1/54.1 KB 14.3 MB/s eta 0:00:00
Collecting pyparsing>=2.0.2
  Downloading pyparsing-3.0.7-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.0/98.0 KB 20.7 MB/s eta 0:00:00
Collecting ptyprocess>=0.5
  Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 KB 14.7 MB/s eta 0:00:00
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.7/138.7 KB 24.2 MB/s eta 0:00:00
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting filelock<4,>=3.2
  Downloading filelock-3.4.2-py3-none-any.whl (9.9 kB)
Collecting distlib<1,>=0.3.1
  Downloading distlib-0.3.4-py2.py3-none-any.whl (461 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 461.2/461.2 KB 51.9 MB/s eta 0:00:00
Collecting platformdirs<3,>=2
  Downloading platformdirs-2.4.1-py3-none-any.whl (14 kB)
Collecting cryptography>=2.0
  Downloading cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 63.8 MB/s eta 0:00:00
Collecting cffi>=1.12
  Downloading cffi-1.15.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (444 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 444.3/444.3 KB 30.0 MB/s eta 0:00:00
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 32.3 MB/s eta 0:00:00
Installing collected packages: webencodings, pylev, ptyprocess, pkginfo, msgpack, lockfile, distlib, certifi, virtualenv-clone, urllib3, tomlkit, six, shellingham, pyparsing, pycparser, poetry-core, platformdirs, pexpect, pastel, jeepney, idna, filelock, crashtest, charset-normalizer, cachy, virtualenv, requests, packaging, html5lib, clikit, cffi, requests-toolbelt, pipenv, cryptography, cleo, cachecontrol, SecretStorage, keyring, poetry
Successfully installed SecretStorage-3.3.1 cachecontrol-0.12.10 cachy-0.3.0 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cleo-0.8.1 clikit-0.6.2 crashtest-0.3.1 cryptography-36.0.1 distlib-0.3.4 filelock-3.4.2 html5lib-1.1 idna-3.3 jeepney-0.7.1 keyring-21.8.0 lockfile-0.12.2 msgpack-1.0.3 packaging-20.9 pastel-0.2.1 pexpect-4.8.0 pipenv-2022.1.8 pkginfo-1.8.2 platformdirs-2.4.1 poetry-1.1.12 poetry-core-1.0.7 ptyprocess-0.7.0 pycparser-2.21 pylev-1.4.0 pyparsing-3.0.7 requests-2.27.1 requests-toolbelt-0.9.1 shellingham-1.4.0 six-1.16.0 tomlkit-0.8.0 urllib3-1.26.8 virtualenv-20.13.0 virtualenv-clone-0.5.7 webencodings-0.5.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Removing intermediate container 16c650c9a013
 ---> 0bc6d930f4e0
Step 8/8 : CMD [ "python" ]
 ---> Running in f889f79064b1
Removing intermediate container f889f79064b1
 ---> 8ba814a0e72b
Successfully built 8ba814a0e72b
Successfully tagged cdk-5bf8e17a0eeb506ba04d24eda67daebef697aa067d69b13cdc66c501f6b40536:latest
Bundling asset CdkPythonStack/handlingFunction/Code/Stage...
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
jsii.errors.JavaScriptError: 
  Error: Failed to bundle asset CdkPythonStack/handlingFunction/Code/Stage, bundle output is located at /github/workspace/cdk.out/asset.e46407f680efdc5e6249cb50a9f81c5d996affe99241b32077a6b2d6cf11998e-error: Error: docker exited with status 1

The CDK declaration is done like so (using __file__) to guarantee the location is always the same):

        handler_function = PythonFunction(
            self, "handlingFunction",
            entry=os.path.join(os.path.dirname(
                os.path.realpath(__file__)), "../src/lambda"),
            runtime=Runtime.PYTHON_3_9,
            index="lambda_function.py",
            environment={
                "LOGLEVEL": "DEBUG"
            },
            timeout=Duration.seconds(20),
            memory_size=512,
            tracing=Tracing.ACTIVE
        )

I know nodeJS functions can be bundled using esbuild but the python functions don't support local bundling.

rileydakota commented 2 years ago

I also encountered the same issue using the PythonFunction construct

sbritocorral commented 1 year ago

Also for me same issue