Lambda Layer Numpy build process?

[ctodd]

Can you share the process for building the Lambda layers? I'd like to update the Lambda runtime to a later version.

I used the AWS lambda/python Docker image and used pip to install the packages into a layer using arm64 as the built target. At first I thought this was caused by missing numpy C libraries, but I've verified they are there.

When running the app locally I get 502 error, and looking in the Lambda logs I get an error when calling my Lambda:

[ERROR] Runtime.ImportModuleError: Unable to import module 'AIMessageProcessor': Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python interpreter from there.

I've seen just a few references to this error, but don't seem to be finding the cause in my case. I figure you might have already won this battle :-)

Kudos to this guy for simplifying the layer build

[KBB99]

Yes, here's the process I used to create a layer with the latest Langchain build, boto3 package, and Python3.12 that I will be publishing shortly: Go into Amazon Codebuild. Pick the runtime you are trying to build the Lambda Layer for. Set the source as none and in the buildspec add the following:

version: 0.2

phases:
  build:
    commands:
      - pip config set global.target ""
      - mkdir -p package/python
      - pip install --target package/python boto3
      - pip install --target package/python numpy
      - pip install --target package/python langchain
      - cd package && zip ../package.zip * -r

artifacts:
  files:
    - 'package.zip'

For the artifact location pick the S3 bucket where you want the created zip file to go. Then just download that zip and use it to create the Lambda Layer.

[jarrillaga]

Hey @KBB99, I could run your script in AWS CodeBuilder and I created package.zip in S3, then I created the layer successfully using this zip and I associated it to my Lambda function.

version: 0.2

phases:
  build:
    commands:
      - pip config set global.target ""
      - mkdir -p package/python
      - pip install --target package/python boto3
      - pip install --target package/python numpy
      - pip install --target package/python langchain
      - cd package && zip ../package.zip * -r

artifacts:
  files:
    - 'package.zip'

Now I'm coding the lambda function and I'm trying to make these imports:

from langchain import LLMChain, SagemakerEndpoint, prompts
from langchain.llms.sagemaker_endpoint import LLMContentHandler

I'm getting this error when executing the lambda:

Response
{
  "errorMessage": "Unable to import module 'lambda_function': Error importing numpy: you should not try to import numpy from\n        its source directory; please exit the numpy source tree, and relaunch\n        your python interpreter from there.",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "082be2a6-7034-4127-ba34-fc159d77ce88",
  "stackTrace": []
}

Any help would be great ! Thanks :)

The problem was that I was using Python3.12. Right now, I'm using to 3.11, I could fix it and it's working like a charm.

[KBB99]

Great to hear @jarrillaga. The layer is runtime specific so the Lambda runtime must match the runtime you used to create it. Please let me know if you have any further questions.