search

terraform-aws-modules / terraform-aws-lambda

Terraform module, which takes care of a lot of AWS Lambda/serverless tasks (build dependencies, packages, updates, deployments) in countless combinations 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/lambda/aws
Apache License 2.0
886 stars 656 forks source link

How to use Poetry with `src` directory but with `poetry.lock` and `pyproject.toml` at root. #524

Closed n8felton closed 5 months ago

n8felton commented 6 months ago

Description

I'm unable to determine how to use the poetry_install = true feature when poetry.lock and pyproject.toml are found in the root directory, but the source files are in the src directory. I believe the problem stems from wanting to keep these files in the root directory, but for the build process to not include any other files from the root directory.

As an example, this can be quickly replicated as such.

poetry new --src aws_lambda
cd aws_lambda
mkdir terraform
poetry add aws-lambda-powertools requests

Then add the Dockerfile to the root, and the Terraform main.tf to the terraform sub-directory.

Dockerfile ``` Dockerfile FROM public.ecr.aws/sam/build-python3.12:latest-arm64 ENV PYTHONUNBUFFERED=1 \ PIP_DISABLE_PIP_VERSION_CHECK=on \ POETRY_HOME="/opt/poetry" \ POETRY_NO_INTERACTION=1 \ POETRY_VIRTUALENVS_CREATE=false \ VIRTUAL_ENV="/venv" ENV PATH="$POETRY_HOME/bin:$VIRTUAL_ENV/bin:$PATH" RUN dnf -y update && \ dnf -y install python3-pip zip && \ dnf -y clean all RUN --mount=type=cache,target=/root/.cache \ curl -sSL https://install.python-poetry.org | python - ```
main.tf ```hcl module "aws_lambda" { source = "terraform-aws-modules/lambda/aws" source_path = [ { path = "..", patterns = [ "!.*", "poetry.lock", "pyproject.toml" ], poetry_install = true }, { path = "../src" } ] create_function = false build_in_docker = true runtime = "python3.12" docker_image = "build-python3.12-poetry:latest-arm64" docker_file = "../Dockerfile" } ```

The final directory structure looks like this

.
├── Dockerfile
├── README.md
├── poetry.lock
├── pyproject.toml
├── src
│   └── aws_lambda
│       └── __init__.py
├── terraform
│   └── main.tf
└── tests
    └── __init__.py

To run the example:

cd terraform
terraform init && terraform apply
Terraform Output ```bash module.aws_lambda.local_file.archive_plan[0]: Creating... module.aws_lambda.local_file.archive_plan[0]: Creation complete after 0s [id=3b113e0cb43e05ce53f0b61194c42b0b236b1fa8] module.aws_lambda.null_resource.archive[0]: Creating... module.aws_lambda.null_resource.archive[0]: Provisioning with 'local-exec'... module.aws_lambda.null_resource.archive[0] (local-exec): Executing: ["python3" ".terraform/modules/aws_lambda/package.py" "build" "--timestamp" "1704229079313553000" "builds/86229c92904b6af70bc3b3f6e4cdc391553f033203b45932d831ebb2e67631f9.plan.json"] module.aws_lambda.null_resource.archive[0] (local-exec): zip: creating 'builds/86229c92904b6af70bc3b3f6e4cdc391553f033203b45932d831ebb2e67631f9.zip' archive module.aws_lambda.null_resource.archive[0] (local-exec): > docker images '--format={{.ID}}' build-python3.12-poetry:latest-arm64 module.aws_lambda.null_resource.archive[0] (local-exec): Installing python dependencies with poetry & pip: ../poetry.lock module.aws_lambda.null_resource.archive[0] (local-exec): > mktemp -d terraform-aws-lambda-XXXXXXXX # /var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-s9qctk_n module.aws_lambda.null_resource.archive[0] (local-exec): Using poetry lock file: ../poetry.lock module.aws_lambda.null_resource.archive[0] (local-exec): > cd /var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-s9qctk_n module.aws_lambda.null_resource.archive[0] (local-exec): > docker run --rm -w /var/task -v /private/var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-s9qctk_n:/var/task:z -v /Users/Nathan.Felton/.ssh/known_hosts:/root/.ssh/known_hosts:z --entrypoint '' 86acf56fe036 /bin/sh -c 'poetry config --no-interaction virtualenvs.create true && poetry config --no-interaction virtualenvs.in-project true && poetry export --format requirements.txt --output requirements.txt --with-credentials && python3.12 -m pip install --no-compile --no-deps --prefix= --target=. --requirement=requirements.txt && chown -R 502:20 .' module.aws_lambda.null_resource.archive[0] (local-exec): Warning: poetry-plugin-export will not be installed by default in a future version of Poetry. module.aws_lambda.null_resource.archive[0] (local-exec): In order to avoid a breaking change and make your automation forward-compatible, please install poetry-plugin-export explicitly. See https://python-poetry.org/docs/plugins/#using-plugins for details on how to install a plugin. module.aws_lambda.null_resource.archive[0] (local-exec): To disable this warning run 'poetry config warnings.export false'. module.aws_lambda.null_resource.archive[0] (local-exec): Collecting aws-lambda-powertools==2.30.2 (from -r requirements.txt (line 1)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading aws_lambda_powertools-2.30.2-py3-none-any.whl (671 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 671.7/671.7 kB 6.7 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting certifi==2023.11.17 (from -r requirements.txt (line 4)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading certifi-2023.11.17-py3-none-any.whl (162 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 12.2 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting charset-normalizer==3.3.2 (from -r requirements.txt (line 7)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (137 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.3/137.3 kB 43.6 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting idna==3.6 (from -r requirements.txt (line 98)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading idna-3.6-py3-none-any.whl (61 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.6/61.6 kB 25.9 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting requests==2.31.0 (from -r requirements.txt (line 101)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading requests-2.31.0-py3-none-any.whl (62 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 24.1 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting typing-extensions==4.9.0 (from -r requirements.txt (line 104)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB) module.aws_lambda.null_resource.archive[0] (local-exec): Collecting urllib3==2.1.0 (from -r requirements.txt (line 107)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading urllib3-2.1.0-py3-none-any.whl (104 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 29.6 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Installing collected packages: urllib3, typing-extensions, requests, idna, charset-normalizer, certifi, aws-lambda-powertools module.aws_lambda.null_resource.archive[0] (local-exec): Successfully installed aws-lambda-powertools-2.30.2 certifi-2023.11.17 charset-normalizer-3.3.2 idna-3.6 requests-2.31.0 typing-extensions-4.9.0 urllib3-2.1.0 module.aws_lambda.null_resource.archive[0] (local-exec): WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: poetry.lock module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: pyproject.toml module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): Created: builds/86229c92904b6af70bc3b3f6e4cdc391553f033203b45932d831ebb2e67631f9.zip module.aws_lambda.null_resource.archive[0]: Creation complete after 3s [id=8773389239142975204] ```

The module successfully finds the poetry files and does the necessary build in docker, but none of the packages installed with pip were included in the zip file.

If I move the poetry.lock and pyproject.toml into the src directory, and change the source_path to "../src", things work as expected, but this is not the desired structure for the project.

Working (but undesired) example ``` . ├── Dockerfile ├── README.md ├── src │ ├── aws_lambda │ │ └── __init__.py │ ├── poetry.lock │ └── pyproject.toml ├── terraform │ └── main.tf └── tests └── __init__.py ``` ```hcl module "aws_lambda" { source = "terraform-aws-modules/lambda/aws" source_path = "../src" create_function = false build_in_docker = true runtime = "python3.12" docker_image = "build-python3.12-poetry:latest-arm64" docker_file = "../Dockerfile" } ``` ``` module.aws_lambda.local_file.archive_plan[0]: Creating... module.aws_lambda.local_file.archive_plan[0]: Creation complete after 0s [id=570299fe8cffcc1e3a422081e9428751c7191fdd] module.aws_lambda.null_resource.archive[0]: Creating... module.aws_lambda.null_resource.archive[0]: Provisioning with 'local-exec'... module.aws_lambda.null_resource.archive[0] (local-exec): Executing: ["python3" ".terraform/modules/aws_lambda/package.py" "build" "--timestamp" "1704230110724126200" "builds/3badf2083eca1de70653d506705d5c7cae12b69bfb5ea233d40f492b9475e443.plan.json"] module.aws_lambda.null_resource.archive[0] (local-exec): zip: creating 'builds/3badf2083eca1de70653d506705d5c7cae12b69bfb5ea233d40f492b9475e443.zip' archive module.aws_lambda.null_resource.archive[0] (local-exec): > docker images '--format={{.ID}}' build-python3.12-poetry:latest-arm64 module.aws_lambda.null_resource.archive[0] (local-exec): Installing python dependencies with poetry & pip: ../src/poetry.lock module.aws_lambda.null_resource.archive[0] (local-exec): > mktemp -d terraform-aws-lambda-XXXXXXXX # /var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-gsvjsawy module.aws_lambda.null_resource.archive[0] (local-exec): Using poetry lock file: ../src/poetry.lock module.aws_lambda.null_resource.archive[0] (local-exec): > cd /var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-gsvjsawy module.aws_lambda.null_resource.archive[0] (local-exec): > docker run --rm -w /var/task -v /private/var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-gsvjsawy:/var/task:z -v /Users/Nathan.Felton/.ssh/known_hosts:/root/.ssh/known_hosts:z --entrypoint '' 86acf56fe036 /bin/sh -c 'poetry config --no-interaction virtualenvs.create true && poetry config --no-interaction virtualenvs.in-project true && poetry export --format requirements.txt --output requirements.txt --with-credentials && python3.12 -m pip install --no-compile --no-deps --prefix= --target=. --requirement=requirements.txt && chown -R 502:20 .' module.aws_lambda.null_resource.archive[0] (local-exec): Warning: poetry-plugin-export will not be installed by default in a future version of Poetry. module.aws_lambda.null_resource.archive[0] (local-exec): In order to avoid a breaking change and make your automation forward-compatible, please install poetry-plugin-export explicitly. See https://python-poetry.org/docs/plugins/#using-plugins for details on how to install a plugin. module.aws_lambda.null_resource.archive[0] (local-exec): To disable this warning run 'poetry config warnings.export false'. module.aws_lambda.null_resource.archive[0] (local-exec): Collecting aws-lambda-powertools==2.30.2 (from -r requirements.txt (line 1)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading aws_lambda_powertools-2.30.2-py3-none-any.whl (671 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 671.7/671.7 kB 9.7 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting certifi==2023.11.17 (from -r requirements.txt (line 4)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading certifi-2023.11.17-py3-none-any.whl (162 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 32.4 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting charset-normalizer==3.3.2 (from -r requirements.txt (line 7)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (137 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.3/137.3 kB 31.3 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting idna==3.6 (from -r requirements.txt (line 98)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading idna-3.6-py3-none-any.whl (61 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.6/61.6 kB 19.2 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting requests==2.31.0 (from -r requirements.txt (line 101)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading requests-2.31.0-py3-none-any.whl (62 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 26.0 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Collecting typing-extensions==4.9.0 (from -r requirements.txt (line 104)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB) module.aws_lambda.null_resource.archive[0] (local-exec): Collecting urllib3==2.1.0 (from -r requirements.txt (line 107)) module.aws_lambda.null_resource.archive[0] (local-exec): Downloading urllib3-2.1.0-py3-none-any.whl (104 kB) module.aws_lambda.null_resource.archive[0] (local-exec): ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 34.5 MB/s eta 0:00:00 module.aws_lambda.null_resource.archive[0] (local-exec): Installing collected packages: urllib3, typing-extensions, requests, idna, charset-normalizer, certifi, aws-lambda-powertools module.aws_lambda.null_resource.archive[0] (local-exec): Successfully installed aws-lambda-powertools-2.30.2 certifi-2023.11.17 charset-normalizer-3.3.2 idna-3.6 requests-2.31.0 typing-extensions-4.9.0 urllib3-2.1.0 module.aws_lambda.null_resource.archive[0] (local-exec): WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding content of directory: /var/folders/9q/tr37p2zx3_59kb6g8j3cj70r0000gp/T/terraform-aws-lambda-gsvjsawy module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: THIRD-PARTY-LICENSES module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requirements.txt module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/package_logger.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/py.typed module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/api_gateway.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/appsync.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/bedrock_agent.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/content_types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/lambda_function_url.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/router.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/util.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/vpc_lattice.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/middlewares/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/middlewares/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/middlewares/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/middlewares/openapi_validation.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/middlewares/schema_validation.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/compat.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/constants.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/dependant.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/encoders.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/models.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/params.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/pydantic_loader.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/swagger_ui/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/swagger_ui/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/swagger_ui/html.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/swagger_ui/swagger-ui-bundle.min.js module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/event_handler/openapi/swagger_ui/swagger-ui.min.css module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/exceptions/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/exceptions/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/compat.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/correlation_paths.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/filters.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/formatter.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/lambda_context.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/logger.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/utils.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/formatters/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/formatters/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/logging/formatters/datadog.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/functions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/metric.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/metrics.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cold_start.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/cloudwatch.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/constants.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/metric_properties.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/cloudwatch_emf/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/datadog/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/datadog/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/datadog/datadog.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/datadog/metrics.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/metrics/provider/datadog/warnings.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/middleware_factory/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/middleware_factory/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/middleware_factory/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/middleware_factory/factory.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/cache_dict.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/constants.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/cookies.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/functions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/headers_serializer.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/json_encoder.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/lazy_import.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/user_agent.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/shared/version.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/tracing/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/tracing/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/tracing/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/tracing/extensions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/tracing/tracer.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/serialization.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/constants.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/provider/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/provider/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/provider/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/provider/kms/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/provider/kms/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/_data_masking/provider/kms/aws_encryption_sdk.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/decorators.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/sqs_fifo_partial_processor.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/batch/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/active_mq_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/alb_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/api_gateway_authorizer_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/api_gateway_proxy_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/appsync_authorizer_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/appsync_resolver_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/aws_config_rule_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/bedrock_agent_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/cloud_watch_custom_widget_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/cloud_watch_logs_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/code_pipeline_job_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/cognito_user_pool_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/common.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/connect_contact_flow_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/dynamo_db_stream_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/event_bridge_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/event_source.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/kafka_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/kinesis_firehose_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/kinesis_stream_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/lambda_function_url_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/rabbit_mq_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/s3_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/s3_object_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/secrets_manager_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/ses_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/shared_functions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/sns_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/sqs_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/vpc_lattice.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/appsync/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/appsync/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/data_classes/appsync/scalar_types_utils.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/appconfig.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/comparators.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/feature_flags.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/feature_flags/schema.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/config.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/idempotency.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/persistence/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/persistence/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/persistence/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/persistence/dynamodb.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/custom_dict.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/dataclass.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/no_op.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/idempotency/serialization/pydantic.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/jmespath_utils/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/jmespath_utils/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/jmespath_utils/envelopes.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/appconfig.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/dynamodb.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/secrets.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/ssm.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parameters/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/compat.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/parser.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/pydantic.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/types.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/apigw.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/apigwv2.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/bedrock_agent.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/cloudwatch.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/dynamodb.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/event_bridge.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/kafka.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/kinesis.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/kinesis_firehose.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/lambda_function_url.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/sns.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/sqs.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/vpc_lattice.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/envelopes/vpc_latticev2.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/alb.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/apigw.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/apigwv2.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/bedrock_agent.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/cloudformation_custom_resource.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/cloudwatch.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/dynamodb.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/event_bridge.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/kafka.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/kinesis.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/kinesis_firehose.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/kinesis_firehose_sqs.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/lambda_function_url.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/s3.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/s3_event_notification.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/s3_object_event.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/ses.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/sns.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/sqs.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/vpc_lattice.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/parser/models/vpc_latticev2.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/_s3_seekable_io.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/compat.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/s3_object.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/transformations/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/transformations/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/transformations/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/transformations/csv.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/transformations/gzip.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/streaming/transformations/zip.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/typing/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/typing/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/typing/lambda_client_context.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/typing/lambda_client_context_mobile_client.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/typing/lambda_cognito_identity.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/typing/lambda_context.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/validation/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/validation/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/validation/base.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/validation/envelopes.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/validation/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools/utilities/validation/validator.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/LICENSE module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda_powertools-2.30.2.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: bin/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: bin/normalizer module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi/__main__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi/cacert.pem module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi/core.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi/py.typed module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/LICENSE module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: certifi-2023.11.17.dist-info/top_level.txt module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/__main__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/api.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/cd.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/constant.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/legacy.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/md.cpython-312-aarch64-linux-gnu.so module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/md.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/md__mypyc.cpython-312-aarch64-linux-gnu.so module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/models.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/py.typed module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/utils.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/version.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/cli/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/cli/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer/cli/__main__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/LICENSE module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/entry_points.txt module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: charset_normalizer-3.3.2.dist-info/top_level.txt module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/codec.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/compat.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/core.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/idnadata.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/intranges.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/package_data.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/py.typed module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna/uts46data.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/LICENSE.md module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: idna-3.6.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/__version__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/_internal_utils.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/adapters.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/api.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/auth.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/certs.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/compat.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/cookies.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/help.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/hooks.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/models.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/packages.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/sessions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/status_codes.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/structures.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests/utils.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/LICENSE module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: requests-2.31.0.dist-info/top_level.txt module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/LICENSE module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: typing_extensions-4.9.0.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/_base_connection.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/_collections.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/_request_methods.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/_version.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/connection.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/connectionpool.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/exceptions.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/fields.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/filepost.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/poolmanager.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/py.typed module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/response.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/contrib/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/contrib/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/contrib/pyopenssl.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/contrib/socks.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/connection.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/proxy.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/request.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/response.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/retry.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/ssl_.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/ssl_match_hostname.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/ssltransport.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/timeout.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/url.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/util.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3/util/wait.py module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/INSTALLER module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/METADATA module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/RECORD module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/REQUESTED module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/WHEEL module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/licenses/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/licenses/LICENSE.txt module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding content of directory: ../src module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: poetry.lock module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: pyproject.toml module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda/ module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: aws_lambda/__init__.py module.aws_lambda.null_resource.archive[0] (local-exec): Created: builds/3badf2083eca1de70653d506705d5c7cae12b69bfb5ea233d40f492b9475e443.zip module.aws_lambda.null_resource.archive[0]: Creation complete after 6s [id=2060810492806018253] ```

Versions

mar-pan commented 6 months ago

Happens to me also! Wonder what is the best way to solve it without moving poetry files. I was trying to use pip_tmp_dir but it does not work with poetry_install

pdecat commented 5 months ago

Hi, I'll try and look into this.

FWIW I'm using this feature with layers and without build_in_docker=true which may explain why I did not face such issue yet, e.g.:

module "lambda-cleanup-layer" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "4.18.0"

  create_layer = true
  layer_name   = "cleanup"

  runtime = "python3.11"

  source_path = [{
    path           = "${path.module}/lambdas/cleanup/",
    poetry_install = true,
    prefix_in_zip  = "python",
    patterns = [
      "!(.*/)?src(/.*)?",
      "!(.*/)?tests(/.*)?",
      "!(.*/)?.venv(/.*)?",
      "!(.*/)?.tox(/.*)?",
      "!(.*/)?.pytest_cache(/.*)?",
    ]
  }]

  artifacts_dir = "${path.root}/.terraform/lambda-cleanup-layer-builds/"

  store_on_s3              = true
  s3_bucket                = "mybucket"
  s3_object_storage_class  = "INTELLIGENT_TIERING"
  recreate_missing_package = false
}

module "lambda-cleanup" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "4.18.0"

  function_name = "cleanup"
  description   = "Lambda function to cleanup stuff"
  handler       = "cleanup.main.lambda_handler"

  runtime = "python3.11"

  layers = [
    module.lambda-cleanup-layer.lambda_layer_arn,
  ]

  source_path = [{
    path = "${path.module}/lambdas/cleanup/src/",
    patterns = [
      "!(.*/)?__pycache__(/.*)?",
    ]
  }]

  artifacts_dir = "${path.root}/.terraform/lambda-cleanup-builds/"

  store_on_s3              = true
  s3_bucket                = "mybucket"
  s3_object_storage_class  = "INTELLIGENT_TIERING"
  recreate_missing_package = false

  cloudwatch_logs_retention_in_days = 30
}

Note: in my case, the terraform files are at the root, the lambdas (I have several of the in the same repository) are in subfolders:

.
├── lambda-cleanup.tf
├── lambda-setup.tf
├── lambdas
│   ├── cleanup
│   │   ├── README.md
│   │   ├── poetry.lock
│   │   ├── pyproject.toml
│   │   ├── src
│   │   │   └── aws_lambda
│   │   │        └── __init__.py
│   │   └── tests
│   │        └── __init__.py
│   ├── setup
...

I also have another use case where the terraform file is in the lambda folder:

.
├── main.tf
├── lambdas
│   ├── cleanup
│   │   ├── lambda.tf
│   │   ├── README.md
│   │   ├── poetry.lock
│   │   ├── pyproject.toml
│   │   ├── src
│   │   │   └── aws_lambda
│   │   │        └── __init__.py
│   │   └── tests
│   │        └── __init__.py
│   ├── setup
...
pdecat commented 5 months ago

@n8felton Re-reading your description of the issue, I believe the issue is caused by the "!.*", pattern in your source_path, it excludes all the installed dependencies.

pdecat commented 5 months ago

As a quick test, does the following work?

module "aws_lambda" {
  source = "terraform-aws-modules/lambda/aws"

  source_path = [
    {
      path = "..",
      patterns = [
        ".*",
        "poetry.lock",
        "pyproject.toml"
      ],
      poetry_install = true
    },
    {
      path = "../src"
    }
  ]

  create_function = false

  build_in_docker = true
  runtime         = "python3.12"
  docker_image    = "build-python3.12-poetry:latest-arm64"
  docker_file     = "../Dockerfile"
}

PS: I know it will embed too many things in the zip, but it is just for the purpose of the test.

n8felton commented 5 months ago

As a quick test, does the following work?

Yes, this works "as expected" and includes the dependancies gathered from the Docker container.

Snip showing it including the urllib3 dependency that is captured from the Docker container, as well as the src directory, but also the undesired terraform directory

[...]
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/WHEEL
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/licenses/
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: urllib3-2.1.0.dist-info/licenses/LICENSE.txt
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: Dockerfile
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: README.md
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: poetry.lock
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: pyproject.toml
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: src/
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: src/aws_lambda/
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: src/aws_lambda/__init__.py
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: terraform/
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: terraform/.terraform.lock.hcl
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: terraform/.terraform.tfstate.lock.info
module.aws_lambda.null_resource.archive[0] (local-exec): zip: adding: terraform/main.tf
[...]
pdecat commented 5 months ago

Now, you just need to exclude what you certainly do not want in the zip archive, e.g.:

module "aws_lambda" {
  source = "terraform-aws-modules/lambda/aws"

  source_path = [
    {
      path = "..",
      patterns = [
        "!Dockerfile",
        "!README.md",
        "!src",
        "!terraform",
      ],
      poetry_install = true
    },
    {
      path = "../src"
    }
  ]

  create_function = false

  build_in_docker = true
  runtime         = "python3.12"
  docker_image    = "build-python3.12-poetry:latest-arm64"
  docker_file     = "../Dockerfile"
}
n8felton commented 5 months ago

The problem with that approach is that I would need to add every file and/or directory I would want to exclude, and it leaves the project open to potentially including files that are added to the repo, but are not added to the exclude list.

The goal is to be explicit in what I want included, not what I want excluded.

pdecat commented 5 months ago

Ok, so https://github.com/terraform-aws-modules/terraform-aws-lambda/pull/525/ is not a fix but a feat.

antonbabenko commented 5 months ago

@pdecat Thank you for the review (so far)!

You can now edit/close issues and PRs in terraform-aws-modules, so feel free to take more ownership and do what you think is necessary. Please mention me in the comments if you think the PR is ready for merge or if you have any questions.

mar-pan commented 5 months ago

Having the same issues as @n8felton

n8felton commented 5 months ago

Ok, so #525 is not a fix but a feat.

I think it's a bug that the pattern filter is affecting dependancies generated by the build_in_docker feature, but only when defining !.*. For example, if I do something like

   source_path = [
    {
      path = "..",
      patterns = [
        "!urllib3/",
        "poetry.lock",
        "pyproject.toml"
      ],
      poetry_install = true
    },
    {
      path = "../src"
    }
  ]

The urllib3 directory generated inside the docker container is still included in the zip.

Ideally, I should be able to define !.* in the pattern and not have it affect any of the dependencies built in docker, it should only apply to the actual path defined in path = ".."

Having said that, #525 does not fix that bug, but rather allows me to more explicitly define where the pyproject.toml file is, similar to how you can explicitly define where the requirements.txt file is for the pip_requirements feature.

pdecat commented 5 months ago

The urllib3 directory generated inside the docker container is still included in the zip.

Hmm, that is surprising.

pdecat commented 5 months ago

What about the pip install method, does it behave in the same way?

pdecat commented 5 months ago

The urllib3 directory generated inside the docker container is still included in the zip.

Ok, to exclude that directory, the pattern should be !somedir/.*, e.g.:

  source_path = [
    {
      path = "${path.module}/../fixtures/python3.9-app-poetry"
      patterns = [
        "!colorful/.*",
        "poetry.lock",
        "pyproject.toml"
      ],
      poetry_install = true
    }
  ]

Log with this pattern:

module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: requirements.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/INSTALLER
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/LICENSE
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/METADATA
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/RECORD
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/REQUESTED
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/WHEEL
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/top_level.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: ignore_please.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: index.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: poetry.lock
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: poetry.toml
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: pyproject.toml
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: docker/
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: docker/Dockerfile
module.package_dir_poetry.null_resource.archive[0] (local-exec): Created: ./builds/package_dir_poetry/cb83579bbb4b720a244b5fb1815d3302f38df3c62e541edfe9b953d98f553581.zip

With !colorful/, it is indeed included:

module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: requirements.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/__init__.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/ansi.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/colors.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/core.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/styles.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/terminal.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/utils.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/data/
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/data/colornames.json
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful/data/rgb.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/INSTALLER
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/LICENSE
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/METADATA
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/RECORD
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/REQUESTED
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/WHEEL
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: colorful-0.5.4.dist-info/top_level.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: ignore_please.txt
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: index.py
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: poetry.lock
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: poetry.toml
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: pyproject.toml
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: docker/
module.package_dir_poetry.null_resource.archive[0] (local-exec): zip: adding: docker/Dockerfile
module.package_dir_poetry.null_resource.archive[0] (local-exec): Created: ./builds/package_dir_poetry/b18237aeb029f606203c6add82123266288622fdad187f8eedd6b13553e3447b.zip
MScottBlake commented 5 months ago

It's great that there are patterns that we can use to exclude files or directories, but there are times when the file list is dynamic and where this approach does not work. It would be great to be able to explicitly define a list of files or directories to add, including the toml files. As it stands today, the poetry flags only work if the project is structured in a very opinionated way.

pdecat commented 5 months ago

As it stands today, the poetry flags only work if the project is structured in a very opinionated way.

Is that really much different from the way it worked with pip?

Not a troll, I'm really trying to understand why these reports all come at a sudden.

I implemented support for poetry in 2022, and it has been in use by several people (me included) since then.

MScottBlake commented 5 months ago

Is that really much different from the way it worked with pip?

The pip implementation allow you to specify the path to requirements.txt. The poetry implementation only checks for pyproject.toml at the root of the path. You cannot specify its location, so yes, it is much different.

I'm really trying to understand why these reports all come at a sudden.

I don't know about the rest, but I have poetry projects where I also have a requirements.txt file because the poetry integration wasn't working. I never circled back and reported it.

n8felton commented 5 months ago

Not a troll, I'm really trying to understand why these reports all come at a sudden.

I implemented support for poetry in 2022, and it has been in use by several people (me included) since then.

The answer to this is quite simple - we only just started attempting to use this module a few weeks ago.

Our project is a new Python project, so we started it from scratch with some of today's recommended best practices, such as using poetry (instead of pip requirements.txt), and using a src directory.

pdecat commented 5 months ago

Noted. FWIW, I'm using src/ directories too: https://github.com/terraform-aws-modules/terraform-aws-lambda/issues/524#issuecomment-1888851514

antonbabenko commented 5 months ago

This issue has been resolved in version 6.8.0 :tada:

github-actions[bot] commented 4 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.