SAM Debug: failed to build layers with Python

[kbadkowski]

Describe the bug

I have a SAM application that consists of lamda functions and one shared lambda layer. The layer uses a makefile identifier for building a resource within the template. I can't run and debug the application from VS Code - it fails on (from aws toolkit logs):

Running command: (not started) [/usr/local/bin/sam build --build-dir /mnt/c/project/project/.aws-sam/output --template /mnt/c/project/project/template.yaml --use-container --manifest /mnt/c/project/project/.aws-sam/debug-requirements.txt --cached --debug]
...
...
...
Fetching public.ecr.aws/sam/build-python3.8:latest-x86_64 Docker container image......
2022-02-14 12:05:17,274 | Mounting /mnt/c/project/project/ as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from the command line argument
...
executing Make: ['make', '--makefile', '/tmp/samcli/manifest/debug-requirements.txt', 'build-ProjectDependencies']
CustomMakeBuilder:MakeBuild failed
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/custom_make/actions.py", line 87, in execute
    self.subprocess_make.run(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/custom_make/make.py", line 88, in run
    raise MakeExecutionError(message=err.decode("utf8").strip())
aws_lambda_builders.workflows.custom_make.make.MakeExecutionError: Make Failed: make: *** No rule to make target `build-ProjectDependencies'.  Stop.
...

The part with --manifest /mnt/c/project/project/.aws-sam/debug-requirements.txt looks very strange to me. It is added by default from aws toolkit in VS Code and it causes the issue.

Running build directly from the terminal:

sam build --template-file template.yaml --use-container --cached --debug

Gives:

...
executing Make: ['make', '--makefile', '/tmp/samcli/source/Makefile', 'build-ProjectDependencies']
CustomMakeBuilder:MakeBuild succeeded
...

To Reproduce

Steps to reproduce the behavior:

1. Create SAM App with lambda layer with makefile as BuildMethod
2. Create debug configuration
3. Run debug

Expected behavior

SAM app should build and run properly after a click on run and debug in VS Code. Build should succeed similar as from directly the terminal.

Desktop (please complete the following information):

• OS: WSL 2 Ubuntu 20.04
• Visual Studio Code Version: 1.64.2
• AWS Toolkit Version: 1.35

Additional context

Configuration:

{
    "configurations": [
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "API Classifier",
            "invokeTarget": {
                "target": "api",
                "templatePath": "${workspaceFolder}/project/template.yaml",
                "logicalId": "ClassifierLambda"
            },
            "api": {
                "path": "/classifier",
                "httpMethod": "post",
                "payload": {
                    "json": {}
                }
            },
            "sam": {
                "containerBuild": true,
                "buildDir": "${workspaceFolder}/project/.aws-sam",
                "buildArguments": [
                    "--cached",
                    "--debug",
                ],
                "localArguments": [
                    "--env-vars",
                    "${workspaceFolder}/project/EnvironmentVariables/env.json"
                ]
            }
        }
    ]
}

[JadenSimon]

Looks like an incompatibility with how the Toolkit handles debugging Python lambdas. We're basically adding debugpy as a debug-only dependency if it's not already present, however, this doesn't take layers into account. --manifest /mnt/c/project/project/.aws-sam/debug-requirements.txt is happening because of this behavior.

One (temporary) workaround could be adding debugpy to requirements.txt in your lambda projects. This should get the Toolkit to stop adding a manifest option when running SAM CLI.

[kbadkowski]

I tried the workaround - it doesn't work for me (still the same error).

My requirements.txt file:

certifi==2021.10.8
click==8.0.3
elasticsearch==7.10.1
et-xmlfile==1.1.0
jdcal==1.4.1
joblib==1.1.0
nltk==3.5
numpy==1.22.2
openpyxl==3.0.9
pandas==1.4.0
python-dateutil==2.8.2
pytz==2021.3
regex==2022.1.18
scikit-learn==1.0.2
scipy==1.7.3
six==1.16.0
threadpoolctl==3.1.0
tqdm==4.62.3
urllib3==1.26.8
XlsxWriter==3.0.2
python-jose==3.3.0
requests==2.27.1
imbalanced-learn==0.9.0
debugpy==1.5.1

[JadenSimon]

Is that the requirements.txt file in your layer? If so, you'll need a requirements.txt file with debugpy in the lambda directory as well.

I'll dig a bit more into this problem, there may be something I'm missing.

[kbadkowski]

Yes, the requirements.txt I posted is in my layer. I added debugpy to requirements.txt of my lambda directory as well but it still doesn't work. The error has been changed:

[ERROR]: "sam build" failed. Timed out waiting for build.
[WARN]: "sam build" failed: /mnt/c/project/project/template.yaml

[JadenSimon]

@kbadkowski

Looks like the build is taking a long time and causing the toolkit to timeout. Try adjusting this settingaws.samcli.lambda.timeout to a higher value, maybe double or more. The value is in milliseconds.

A fast way to find the setting is by running the command Preferences: Open User Settings then pasting aws.samcli.lambda.timeout into the settings search bar.

Updates regarding timeouts in SAM debug flows are partially tracked apart of this issue: https://github.com/aws/aws-toolkit-vscode/issues/2455

[kbadkowski]

Your bits of advice helped me. I managed to run locally my app using the AWS toolkit. Thank you for the workaround and help. Would be nice if it will work in the future without adding debugpy==1.5.1 to the lambda's requirements.txt.

[minhio]

any update on this issue? will it be fixed?