Bug

I have a FastAPI setup with Odmantic and Mangum. I use Mangum when the API is deployed to AWS Lambda, and when I run the API locally I use "python app.py". Locally, I have no issues reading and writing data to my MongoDB Atlas cluster using Odmantic. When I run the same code in AWS Lambda, I don't get any results from the database, it always returns None. I don't get any errors as well, I am not able to tell if there is an issue with the connection or the AWS Lambda environment.

The AWS Lambda is not in a VPC and I am using Serverless Framework and the Amazon 2 Linux Docker Image to deploy the code.

Current Behavior

Reads using Odmantic on AWS Lambda returns no data and no error message.

Expected behavior

For reads with Odmantic to return an item from the database or raise an error.

Environment

Locally - Everything works as expected. AWS Lambda - I see no item returned or error is thrown.

Code

Poetry

[tool.poetry]
name = "lambda-odmantic"
version = "0.0.1"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "^3.9"
fastapi = "^0.70.0"
odmantic = "^0.3.5"
mangum = "^0.12.3"
gunicorn = "^20.1.0"
httpx = "^0.20.0"
boto3 = "^1.18.61"
pytz = "^2021.3"
PyJWT = "^1.7.1"
hashids = "^1.3.1"
bcrypt = "^3.1.7"
pymongo = {version = "^3.12.0", extras = ["srv", "tls"]}
uvicorn = {version = "^0.15.0", extras = ["standard"]}

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Database Engine Class

import os
from odmantic import AIOEngine
from motor.motor_asyncio import AsyncIOMotorClient

APP_DB_USER = os.environ["APP_DB_USER"]
APP_DB_PASSWORD = os.environ["APP_DB_PASSWORD"]
APP_DB_HOST = os.environ["APP_DB_HOST"]
APP_DB_NAME = os.environ["APP_DB_NAME"]
DB_MAX_CONNECTIONS = os.getenv("DB_MAX_CONNECTIONS", "10")
DB_MIN_CONNECTIONS = os.getenv("DB_MIN_CONNECTIONS", "10")

class DBEngine:
    MONGO_URL = f"mongodb+srv://{APP_DB_USER}:{APP_DB_PASSWORD}@{APP_DB_HOST}/app_db?retryWrites=true&readPreference=nearest"

    def __init__(self) -> None:
        self._client: AsyncIOMotorClient = None
        self._engine: AIOEngine = None

    @property
    def client(self) -> AsyncIOMotorClient:
        return self._client

    @property
    def engine(self) -> AIOEngine:
        return self._engine

    async def connect(self):
        self._client: AsyncIOMotorClient = AsyncIOMotorClient(
            self.MONGO_URL,
            maxPoolSize=DB_MAX_CONNECTIONS,
            minPoolSize=DB_MIN_CONNECTIONS
        )
        self._engine: AIOEngine = AIOEngine(
            motor_client=self._client,
            database=APP_DB_NAME
        )

    async def close(self):
        self._client.close()

db_engine = DBEngine()

Movie Object

from odmantic import Model, Field
from typing import Optional
from datetime import datetime

class Movie(Model):
    name: Optional[str] = Field(default=None)
    producer: Optional[str] = Field(default=None)
    year: Optional[int] = Field(default=None)
    is_deleted: bool = Field(default=False)
    created_by: str
    modified_by: str
    created_at: datetime
    modified_at: datetime

    class Config:
        collection = "movie"
        parse_doc_with_default_factories = True

Resource Class

import json
import traceback
import logging
from fastapi import APIRouter, status
from odmantic import ObjectId
from api.db import db_engine
from api.movie import Movie

router = APIRouter()
LOGGER_FORMAT = "%(asctime)s %(levelname)s %(name)s %(threadName)s : %(message)s)"

logger = logging.getLogger()
logging.basicConfig(level=logging.DEBUG, format=LOGGER_FORMAT)

@router.get("/movies/{movie_id}", response_model=Movie, status_code=status.HTTP_200_OK)
async def get_movie(movie_id: str):
        return await db_engine.engine.find_one(Movie, Movie.id == ObjectId(movie_id))

App.py

import os
import uvicorn
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from mangum import Mangum
from api.db import db_engine

from api import movie_resource as movie_resource_v1

app = FastAPI()

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Configure Routers
app.include_router(movie_resource_v1.router)

# Database Connection
@app.on_event("startup")
async def on_startup():
    await db_engine.connect()

@app.on_event("shutdown")
async def on_shutdown():
    await db_engine.close()

# For Deployment with Serverless Framework
handler = Mangum(app, lifespan="auto")
# Driver
if __name__ == "__main__":
    uvicorn.run("app:app", host="0.0.0.0", port=8080, reload=True, log_level="info")

Serverless Framework Yaml

# Name of the service, you would see this as lambda
service: lambda-odmantic
# Import environment and region appropriate configuration
config: ${file(config/${self:provider.stage}-${self:provider.region}.yml)}
configValidationMode: warn # Modes - error, warn, off.
variablesResolutionMode: 20210326 # Enable complete support for "ssm" variables resolution.
# Configuration for provider, as we are using AWS with python 3.9
provider:
  name: aws
  runtime: python3.9
  stage: ${opt:stage,'dev'}
  region: ${opt:region, 'us-east-1'}
  timeout: 30 # The default is 6 seconds. Note: API Gateway current maximum is 30 seconds
  logRetentionInDays: 14 # Set the default RetentionInDays for a CloudWatch LogGroup
  versionFunctions: false # Optional function versioning
  architecture: x86_64 # Values - x86_64, arm64
  lambdaHashingVersion: 20201221 # optional, version of hashing algorithm that should be used by the framework
  deploymentBucket:
    # name: ${self:config.artifactLambdaBucket}
    maxPreviousDeploymentArtifacts: 3
    blockPublicAccess: true
    serverSideEncryption: AES256
  environment:
    ENV_NAME: ${self:provider.stage}
    REGION: ${self:provider.region}
    EVENT_STREAM_API_ENDPOINT: ${self:config.eventStreamAPIEndpoint}
    SENTRY_ENABLED: ${self:config.enableSentry}
    APP_DB_USER: ${ssm(${self:provider.region}):/${self:provider.stage}/demo/app_db_user}
    APP_DB_PASSWORD: ${ssm(${self:provider.region}):/${self:provider.stage}/demo/app_db_password}
    APP_DB_HOST: ${ssm(${self:provider.region}):/${self:provider.stage}/demo/app_db_host}
    APP_DB_NAME: ${ssm(${self:provider.region}):/${self:provider.stage}/demo/app_db_name}
    DB_MAX_CONNECTIONS: ${self:config.dbMaxConnections}
    DB_MIN_CONNECTIONS: ${self:config.dbMinConnections}
  memorySize: 2048
  apiGateway:
    restApiId: ${self:config.gwRestApiId}
    restApiRootResourceId: ${self:config.gwRestApiRootResourceId}
  tracing:
    lambda: true
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "xray:*"
      Resource: "*"
    - Effect: "Allow"
      Action:
        - "logs:*"
      Resource: "*"
    - Effect: "Allow"
      Action:
        - "ssm:*"
      Resource: "*"
# Plugins which we are using to making our app serverless
plugins:
  - serverless-python-requirements
  - serverless-prune-plugin
# Exclude unneeded packages
package:
  patterns:
    - '!.git/**'
    - '!node_modules/**'
    - '!venv/**'
# Use Linux to build Python Dependencies and create Lambda Layers from there
custom:
  pythonRequirements:
    dockerizePip: false
    slim: true
    slimPatternsAppendDefaults: false
    slimPatterns:
      - "**/*.pyc"
      - "**/*.pyo"
      - "**/__pycache__*"
    invalidateCaches: true
    layer: true
    useStaticCache: false
  stages:
    - dev
    - qa
    - prod
  corsConfig:
    origin: '*' # <-- Specify allowed origin
    headers: # <-- Specify allowed headers
      - Content-Type
      - Accept
      - Accept-Encoding
      - X-Amz-Date
      - Authorization
      - X-Api-Key
      - X-Amz-Security-Token
      - X-Amz-User-Agent
      - X-Correlation-Id
    allowCredentials: false
  prune:
    automatic: true
    includeLayers: true
    number: 3
functions:
  app:
    handler: app.handler
    # reservedConcurrency: ${self:config.reservedConcurrencyCount}
    # provisionedConcurrency: ${self:config.provisionedConcurrencyCount}
    layers:
      - {Ref: PythonRequirementsLambdaLayer}
    description: Handles the following paths - /movies
    events:
      - http:
          path: /movies
          method: ANY
          authorizer: ${self:config.authorizer}
          cors:
            origin: ${self:custom.corsConfig.origin}
            headers: ${self:custom.corsConfig.headers}
            allowCredentials: ${self:custom.corsConfig.allowCredentials}
      - http:
          path: /movies/{proxy+}
          method: ANY
          authorizer: ${self:config.authorizer}
          cors:
            origin: ${self:custom.corsConfig.origin}
            headers: ${self:custom.corsConfig.headers}
            allowCredentials: ${self:custom.corsConfig.allowCredentials}

art049 / odmantic

Unable to read data with Mangum, FastAPI, and Odmantic on AWS Lambda #196

Bug