Miserlou / Zappa

Serverless Python
https://blog.zappa.io/
MIT License
11.89k stars 1.21k forks source link

Add Docker Container Image Support #2188

Closed ian-whitestone closed 2 years ago

ian-whitestone commented 3 years ago

Earlier this month, AWS announced container image support for AWS Lambda. This means you can now package and deploy lambda functions as container images, instead of using zip files. The container image based approach will solve a lot of headaches caused by the zip file approach, particularly with file sizes (container images can be up to 10GB) and the dependency issues we all know & love.

In an ideal end state, you should be able to call zappa deploy / zappa update / zappa package (etc.) and specify whether you want to use the traditional zip-based approach or new Docker container based approach. If choosing the latter, Zappa would automatically:

For a MVP, we should take a BYOI (bring your own image) approach and just get zappa deploy and zappa update to deploy a lambda function using an existing Docker Image that complies with these guidelines.

ian-whitestone commented 3 years ago

Initial Exploration

For an MVP, we should take a BYOI (bring your own image) approach and just get zappa deploy and zappa update to deploy a lambda function using an existing Docker Image that complies with these guidelines.

I was able to get a little POC doing this ☝️ :

image

This is the app.py:

import time

from flask import Flask, jsonify

app = Flask(__name__)

@app.route("/")
def serve():
    return jsonify(success=True)

@app.route("/time")
def get_current_time():
    return {"time": round(time.time())}

Steps

1) I prebuilt a Docker image

FROM amazon/aws-lambda-python:3.8
COPY ./ /var/task/

# Install dependencies with poetry
RUN pip install poetry
RUN POETRY_VIRTUALENVS_CREATE=false poetry install --no-root

CMD [ "handler.lambda_handler" ]

The only hack here is you need to have the handler.py (from zappa library) and zappa_settings.py files baked into your docker image. The zappa_settings.py is read in by the handler.py here, and gets generated here in the zip file creation flow.

To get around this, I'm thinking the following may be a simple solution:

2) I modified the Zappa code to accept -d / --docker-image-uri as a parameter. If this is provided, it skips all the zip compiling steps, and uses the new lambda create_function API to provide the Docker Image URI.

dmarkey commented 3 years ago

This project looks a little dead.

As the workflow is pretty different with docker, maybe worth forking the relevant bits perhaps?

dmarkey commented 3 years ago

Scratch that.. for the use-case I'm describing https://github.com/adamchainz/apig-wsgi seems to suffice.

(I'm thinking more conventional docker build + terraform kinda setup)

siran commented 3 years ago

works like a charm: https://github.com/adamchainz/apig-wsgi

ArtikUA commented 3 years ago

Keep in mind that Lambdas which are created from ECR have very long cold starts: for my project it's 4.5 seconds compared with 1.2 seconds before Looks like we need a feature like "concurrency" for warmups

programmingwithalex commented 2 years ago

@ian-whitestone using the -d / --docker-image-uri flag, is there a way to pass in multiple --build-arg flags for the image?

ian-whitestone commented 2 years ago

Let's move the conversation over to https://github.com/zappa/Zappa/issues/922 (the new repo). I will reply there @alexanderdamiani @ArtikUA !