Lambda error in testing "Runtime.ImportModuleError: Unable to import module..., No Module Name..."

[mtgnoah]

I am trying to transfer an existing project to the lambda infastructure. I have simplified lots of the code here to make it a working example but I have to import many files inside of the shared layer and folders inside of it. I want to use a layer because to help simplify the structure because their is so many packages almost 800mb worth so I don't even know if lambda is the best option or if I should use a different aws service.

The project structure is:

.
|__sam_app
|    |__app
|    |    |__app.py
|    |    |_Dockerfile
|    |    |__requirements.txt
|    |    __shared
|    |        |__python
|    |            |__common.py
|    |        |__requirements.txt
|    |__template.yml

The template.yml file is:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Timeout: 150
    MemorySize: 5000
  Api:
    BinaryMediaTypes:
      - image/png
      - image/jpg
      - image/jpeg

Resources:
  SharedLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      CompatibleRuntimes:
        - python3.9
      ContentUri: app/shared
      Description: Provides the base backend shared library and dependencies
      LayerName: shared-layer
    Metadata:
      BuildMethod: python3.9

  InferenceFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      PackageType: Image
      Layers:
        - !Ref SharedLayer
      Events:
        Inference:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /classify_digit
            Method: post
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./app
      DockerTag: python3.9-v1

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  InferenceApi:
    Description: "API Gateway endpoint URL for Prod stage for Inference function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/classify_digit/"
  InferenceFunction:
    Description: "Inference Lambda Function ARN"
    Value: !GetAtt InferenceFunction.Arn
  InferenceFunctionIamRole:
    Description: "Implicit IAM Role created for Inference function"
    Value: !GetAtt InferenceFunction.Arn

I placed everything into one template file because that is how the template came.

Here is the Dockerfile

FROM public.ecr.aws/lambda/python:3.9

COPY app.py requirements.txt ./

RUN python3.9 -m pip install -r requirements.txt -t .

CMD ["app.lambda_handler"]

I have to run sam build --use-container or I get errors when I am trying to get the python requirements for the layer. To run a test I run sam local invoke InferenceFunction --event events/event.json I get the following output

Invoking Container created from inferencefunction:python3.9-v1
Building image.................
Skip pulling image and use local one: inferencefunction:rapid-1.31.0.

START RequestId: f1362026-ebb5-4153-984a-15ae3e2d99a2 Version: $LATEST
Traceback (most recent call last): Unable to import module 'app': No module named 'common'
END RequestId: f1362026-ebb5-4153-984a-15ae3e2d99a2
REPORT RequestId: f1362026-ebb5-4153-984a-15ae3e2d99a2  Init Duration: 0.40 ms  Duration: 2704.19 ms    Billed Duration: 2800 ms        Memory Size: 5000 MB    Max Memory Used: 5000 MB
{"errorMessage": "Unable to import module 'app': No module named 'common'", "errorType": "Runtime.ImportModuleError", "requestId": "f1362026-ebb5-4153-984a-15ae3e2d99a2", "stackTrace": []}

the app.py is

import torch
import torchvision
import base64
import json
import numpy as np
import cv2
import torch.nn as nn
import torch.nn.functional as F
import os
from PIL import Image
from io import BytesIO

import common as common

def lambda_handler(event, context):
    image_bytes = event['body'].encode('utf-8')

    prediction = common.sum(1,2)

    return {
        'statusCode': 200,
        # 'body': json.dumps(dataSet)
        'body': json.dumps(
            {
                "predicted_label": prediction,
            }
        )
    }

System Info: Windows 10 sam 1.31.0 python 3.9

I'm not sure where the problem is: if it is my folder structure, the number of packages, or even sam itself. I have looked at issue #2335 but it still did not help fix my issues Any help or advice is very appreciated.

[CoshUS]

Hey @mtgnoah, Image typed Lambda Functions do not support Lambda Layers. https://aws.amazon.com/blogs/compute/working-with-lambda-layers-and-extensions-in-container-images/

Lambda functions packaged as container images do not support adding Lambda layers to the function configuration.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.