Bug: Hot Reload not working in VSCode DevContainer with sam local start-api

[fortivi002]

Description:

I am using VSCode DevContainer for my project, and everything works fine except for the hot reload when using sam local start-api. The hot reload feature is not functioning as expected when running the project in the DevContainer environment. Specifically, the browser does not reflect the changes in the code automatically after saving.

Steps to reproduce:

1. Initialize a basic project in the ./project directory by running the following command:

 sam init

You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Data processing
        3 - Hello World Example with Powertools for AWS Lambda
        4 - Multi-step workflow
        5 - Scheduled task
        6 - Standalone function
        7 - Serverless API
        8 - Infrastructure event management
        9 - Lambda Response Streaming
        10 - Serverless Connector Hello World Example
        11 - Multi-step workflow with Connectors
        12 - Full Stack
        13 - Lambda EFS example
        14 - DynamoDB Example
        15 - Machine Learning
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: n

Which runtime would you like to use?
        1 - aot.dotnet7 (provided.al2)
        2 - dotnet6
        3 - go1.x
        4 - go (provided.al2)
        5 - graalvm.java11 (provided.al2)
        6 - graalvm.java17 (provided.al2)
        7 - java17
        8 - java11
        9 - java8.al2
        10 - java8
        11 - nodejs18.x
        12 - nodejs16.x
        13 - nodejs14.x
        14 - nodejs12.x
        15 - python3.9
        16 - python3.8
        17 - python3.7
        18 - python3.10
        19 - ruby3.2
        20 - ruby2.7
        21 - rust (provided.al2)
Runtime: 18

What package type would you like to use?
        1 - Zip
        2 - Image
Package type: 1

Based on your selections, the only dependency manager available is pip.
We will proceed copying the template using pip.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: n

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: n

Project name [sam-app]:

    -----------------------
    Generating application:
    -----------------------
    Name: sam-app
    Runtime: python3.10
    Architectures: x86_64
    Dependency Manager: pip
    Application Template: hello-world
    Output Directory: .
    Configuration file: sam-app\samconfig.toml

    Next steps can be found in the README file at sam-app\README.md

Commands you can use next
=========================
[*] Create pipeline: cd sam-app && sam pipeline init --bootstrap
[*] Validate SAM template: cd sam-app && sam validate
[*] Test Function in the Cloud: cd sam-app && sam sync --stack-name {stack-name} --watch

2. Add the ./project/.devcontainer/devcontainer.json file with the following configuration:

{
    "name": "Python 3",
    "build": {
        "dockerfile": "Dockerfile",
        "context": "..",
        "args": {
            // Update 'VARIANT' to pick a Python version: 3, 3.10, 3.9, 3.8, 3.7, 3.6
            // Append -bullseye or -buster to pin to an OS version.
            // Use -bullseye variants on local on arm64/Apple Silicon.
            "VARIANT": "3.10-bullseye",
            // Options
            "NODE_VERSION": "lts/*"
        }
    },
    // Configure tool-specific properties.
    "customizations": {
        // Configure properties specific to VS Code.
        "vscode": {
            // Set *default* container specific settings.json values on container create.
            "settings": {
                "python.defaultInterpreterPath": "/usr/local/bin/python",
                "python.formatting.provider": "black",
                "python.formatting.": "black",
                "editor.formatOnSave": true,
                "[python]": {
                    "editor.formatOnSave": true
                }
            },
            // Add the IDs of extensions you want installed when the container is created.
            "extensions": [
                "ms-python.python"
            ]
        }
    },
    // Use 'forwardPorts' to make a list of ports inside the container available locally.
    // "forwardPorts": [],
    // Use 'postCreateCommand' to run commands after the container is created.
    // "postCreateCommand": "pip3 install --user -r requirements.txt",

    "features": {
        "ghcr.io/devcontainers/features/docker-in-docker:2": {}
    },
    // Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
    "remoteUser": "vscode",
    "mounts": [
        "source=${localEnv:HOME}${localEnv:USERPROFILE}/.aws,target=/home/vscode/.aws,type=bind"
    ]
}

3. Add the ./project/.devcontainer/Dockerfile file with the following configuration:

   
   # [Choice] Python version (use -bullseye variants on local arm64/Apple Silicon): 3, 3.10, 3.9, 3.8, 3.7, 3.6, 3-bullseye, 3.10-bullseye, 3.9-bullseye, 3.8-bullseye, 3.7-bullseye, 3.6-bullseye, 3-buster, 3.10-buster, 3.9-buster, 3.8-buster, 3.7-buster, 3.6-buster
   ARG VARIANT=3-bullseye
   FROM mcr.microsoft.com/vscode/devcontainers/python:${VARIANT}

[Choice] Node.js version: none, lts/*, 16, 14, 12, 10

ARG NODE_VERSION="none" RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi

[Optional] If your pip requirements rarely change, uncomment this section to add them to the image.

COPY requirements.txt /tmp/pip-tmp/

RUN pip3 --disable-pip-version-check --no-cache-dir install -r /tmp/pip-tmp/requirements.txt \

&& rm -rf /tmp/pip-tmp

[Optional] Uncomment this section to install additional OS packages.

RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \

&& apt-get -y install --no-install-recommends awscli

USER root

RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \ unzip awscliv2.zip && \ sudo ./aws/install

RUN curl -L https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip -o "aws-sam-cli-linux-x86_64.zip" && \ unzip "aws-sam-cli-linux-x86_64.zip" -d sam-installation && \ sudo ./sam-installation/install

[Optional] Uncomment this line to install global node packages.

RUN su vscode -c "source /usr/local/share/nvm/nvm.sh && npm install -g " 2>&1

4. Open the ./project directory in VSCode, and use the command Ctrl+Shift+P to select "Dev Containers: Rebuild and Reopen in Container."
5.  Once the container starts, navigate to the sam-app directory and run sam local start-api.

6. Open a browser and go to localhost:3000/hello to invoke the function.

7. Make changes to the Lambda function code.

8. Save the file.

### Observed result:
The browser does not automatically reflect the changes in the code after saving. The hot reload does not work as expected.
<!-- Please provide command output with `--debug` flag set. -->

vscode ➜ /workspaces/project/sam-app $ sam local start-api --debug 2023-08-02 14:52:32,329 | Config file location: /workspaces/project/sam-app/samconfig.toml
2023-08-02 14:52:32,335 | Loading configuration values from [default.['local', 'start-api'].parameters] (env.command_name.section) in config file at '/workspaces/project/sam-app/samconfig.toml'...
2023-08-02 14:52:32,338 | Configuration values successfully loaded.
2023-08-02 14:52:32,341 | Configuration values are: {'stack_name': 'sam-app', 'warm_containers': 'EAGER'}
2023-08-02 14:52:32,357 | Using SAM Template at /workspaces/project/sam-app/template.yaml
2023-08-02 14:52:32,424 | Using config file: samconfig.toml, config environment: default
2023-08-02 14:52:32,426 | Expand command line arguments to:
2023-08-02 14:52:32,427 | --template_file=/workspaces/project/sam-app/template.yaml --host=127.0.0.1 --port=3000 --static_dir=public --layer_cache_basedir=/home/vscode/.aws-sam/layers-pkg --warm_containers=EAGER
--container_host=localhost --container_host_interface=127.0.0.1
2023-08-02 14:52:32,609 | local start-api command is called
2023-08-02 14:52:32,620 | No Parameters detected in the template
2023-08-02 14:52:32,658 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id
2023-08-02 14:52:32,660 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id
2023-08-02 14:52:32,663 | 0 stacks found in the template
2023-08-02 14:52:32,665 | No Parameters detected in the template
2023-08-02 14:52:32,695 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id
2023-08-02 14:52:32,696 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id
2023-08-02 14:52:32,698 | 2 resources found in the stack
2023-08-02 14:52:32,700 | Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
2023-08-02 14:52:32,703 | --base-dir is not presented, adjusting uri hello_world/ relative to /workspaces/project/sam-app/template.yaml
2023-08-02 14:52:32,713 | watch resource /workspaces/project/sam-app/template.yaml
2023-08-02 14:52:32,714 | Create Observer for resource /workspaces/project/sam-app/template.yaml with recursive True
2023-08-02 14:52:32,721 | watch resource /workspaces/project/sam-app/template.yaml's parent /workspaces/project/sam-app
2023-08-02 14:52:32,723 | Create Observer for resource /workspaces/project/sam-app with recursive False
2023-08-02 14:52:32,732 | Initializing the lambda functions containers.
2023-08-02 14:52:32,734 | Async execution started
2023-08-02 14:52:32,738 | Invoking function functools.partial(<function InvokeContext._initialize_all_functions_containers..initialize_function_container at 0x7f70e53a9c60>,
Function(function_id='HelloWorldFunction', name='HelloWorldFunction', functionname='HelloWorldFunction', runtime='python3.10', memory=None, timeout=3, handler='app.lambda_handler', imageuri=None, packagetype='Zip',
imageconfig=None, codeuri='/workspaces/project/sam-app/hello_world', environment=None, rolearn=None, layers=[], events={'HelloWorld': {'Type': 'Api', 'Properties': {'Path': '/hello', 'Method': 'get', 'RestApiId':
'ServerlessRestApi'}}}, metadata={'SamResourceId': 'HelloWorldFunction'}, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], function_url_config=None,
function_build_info=<FunctionBuildInfo.BuildableZip: ('BuildableZip', 'Regular ZIP function which can be build with SAM CLI')>, stack_path='', runtime_management_config=None))
2023-08-02 14:52:32,746 | Waiting for async results
2023-08-02 14:52:32,753 | No environment variables found for function 'HelloWorldFunction'
2023-08-02 14:52:32,755 | Loading AWS credentials from session with profile 'None'
2023-08-02 14:52:32,789 | Resolving code path. Cwd=/workspaces/project/sam-app, CodeUri=/workspaces/project/sam-app/hello_world
2023-08-02 14:52:32,791 | Resolved absolute path to code is /workspaces/project/sam-app/hello_world
2023-08-02 14:52:32,807 | watch resource /workspaces/project/sam-app/hello_world
2023-08-02 14:52:32,808 | Create Observer for resource /workspaces/project/sam-app/hello_world with recursive True
2023-08-02 14:52:32,816 | watch resource /workspaces/project/sam-app/hello_world's parent /workspaces/project/sam-app
2023-08-02 14:52:32,820 | Code /workspaces/project/sam-app/hello_world is not a zip/jar file
2023-08-02 14:52:33,731 | Local image is up-to-date
2023-08-02 14:52:33,741 | Using local image: public.ecr.aws/lambda/python:3.10-rapid-x86_64.

2023-08-02 14:52:33,743 | Mounting /workspaces/project/sam-app/hello_world as /var/task:ro,delegated, inside runtime container
2023-08-02 14:52:34,500 | Async execution completed
2023-08-02 14:52:34,501 | Containers Initialization is done.
2023-08-02 14:52:34,502 | Found '1' API Events in Serverless function with name 'HelloWorldFunction'
2023-08-02 14:52:34,503 | Detected Inline Swagger definition
2023-08-02 14:52:34,504 | Parsing Swagger document using 2.0 specification
2023-08-02 14:52:34,506 | Lambda function integration not found in Swagger document at path='/hello' method='get'
2023-08-02 14:52:34,507 | Found '0' APIs in resource 'ServerlessRestApi'
2023-08-02 14:52:34,508 | Found '0' authorizers in resource 'ServerlessRestApi'
2023-08-02 14:52:34,510 | Removed duplicates from '0' Explicit APIs and '1' Implicit APIs to produce '1' APIs
2023-08-02 14:52:34,512 | 1 APIs found in the template
2023-08-02 14:52:34,516 | Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET]
2023-08-02 14:52:34,518 | You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected
instantly/automatically. If you used sam build before running local commands, you will need to re-run sam build for the changes to be picked up. You only need to restart SAM CLI if you update your AWS SAM template
2023-08-02 14:52:34,521 | Localhost server is starting up. Multi-threading = True
2023-08-02 14:52:34 WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

• Running on http://127.0.0.1:3000 2023-08-02 14:52:34 Press CTRL+C to quit

Expected result:

I expect the browser to automatically show the changes in the code without the need to stop the process and rerun the sam local start-api command.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: DevContainer Linux
2. sam --version 1.94.0:
3. AWS region: us-east-2

{
  "version": "1.94.0",
  "system": {
    "python": "3.11.3",
    "os": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.31"
  },
  "additional_dependencies": {
    "docker_engine": "23.0.6+azure-2",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

You need to install the Dev Containers extension to VSCode : Dev Containers v0.299.0

[jfuss]

@fortivi002 I am not familiar with DevContainer and how it interacts with the local machine.

Not sure what "I expect the browser to automatically show the changes in the code without the need to stop the process and rerun the sam local start-api command." means? Are you hitting the endpoint again and not seeing the updates?

"hot reloading" works by allows mounting the new files into the container on invoke. So assuming you are not changing the template or the handler path, things should just work. If you are updating the template or handler location or using sam build, you will need to restart as the template is cached for the duration of the command.

[fortivi002]

@jfuss Hi, I apologize if my previous explanation was unclear. Yes, I am hitting the endpoint again, but I don't see the changes taking effect. The issue is that hot-reloading is not working as expected because I'm not modifying the template or the handler path. For example, if I change the message returned by the endpoint and save the file, the updated message is not visible when hitting the endpoint again. To see the changes, I have to stop the process and rerun 'sam local start-api'

[jfuss]

@fortivi002 Does this work when not using DevContainer?

It will take time to setup this on my end, but if it works without it likely something happening in how DevContainer works.

[fortivi002]

@jfuss Yes , it's working when I'm not using a DevContainer.

[jfuss]

@fortivi002 If this works without DevContainer, it's likely this is an issue with your setup.

It looks like you might be starting sam local within the DevContainer, which could lead to docker in docker issues. Things I would try:

1. Run SAM CLI from your host (this is actually the preferred way to run SAM CLI as you don't have to deal with docker in docker issues).
2. Ensure you changes are being sync'ed back to your host machine.

[lucashuy]

Hi @fortivi002, just checking in to see if there are any new updates regarding the steps to try out listed above, and if you are still experiencing problems with this.

[fortivi002]

@lucashuy Hi, the solution provided work. However, my original intention was to get it SAM working within a devcontainer. Using it locally without a devcontainer doesn't align with the scenario I'm aiming to address. If there are any insights or solutions tailored to SAM in a devcontainer, I'd greatly appreciate it.

[mndeveci]

Thanks for creating this issue @fortivi002

I've followed your steps to re-produce the issue, but I was able to start dev-container, start sam local start-api and I was able to invoke my function. I made changes through VSCode and curled endpoint again and I was able to see my changes. I even tested by changing the source file through another editor and I was still able to see the change when I recurled the endpoint.

Couple of things to confirm;

1. Are you refreshing your browser page after you update your function? For this case, I would even recommend using something like curl.
2. If you are running sam build before running sam local start-api, any changes to source files will not be picked up by sam local start-api process. You need to re-build and re-start local api process in order changes to take effect.

[moelasmar]

Closing due to inactivity, please create another issue if you still have the same problem, or any other problems.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.