Unable to use New Relic layer in lambda that is already using the aws-lambda-web-adapter layer

[lucaswehmuth]

Description

The lambda runtime is unable to locate the _newrelic_lambdawrapper.handler in a lambda that uses the aws-lambda-web-adapter layer (https://github.com/awslabs/aws-lambda-web-adapter) which apparently is another wrapper.

So the issue is trying to set a wrapper inside another wrapper.

Steps to Reproduce

Define a lambda resource using the Serverless Application Model (SAM) like the one below:

  LambdaRestApiWebhooksAuth:
    Type: AWS::Serverless::Function
    DependsOn: RestApiWebhooksAuth
    Properties:
      CodeUri: ./src/handler
      Handler: newrelic_lambda_wrapper.handler
      FunctionName: !FindInMap [StageMappings, !Ref StageName, lambdaNameHandler]
      Role: !Sub "arn:aws:iam::${AWS::AccountId}:role/role-lambda-api"
      Architectures:
        - arm64
      Runtime: python3.10
      MemorySize: 128
      Environment:
        Variables:
          AWS_LAMBDA_EXEC_WRAPPER: /opt/bootstrap
          PORT: "8000"
          ENV: !Ref StageName
          NEW_RELIC_LAMBDA_HANDLER: run.sh
          NEW_RELIC_ACCOUNT_ID: "<ACCOUNT_ID>"
          NEW_RELIC_TRUSTED_ACCOUNT_KEY: "<ACCOUNT_ID>"
          NEW_RELIC_LICENSE_KEY: "<API_KEY>"
      Layers:
        - !Sub arn:aws:lambda:${AWS::Region}:753240598075:layer:LambdaAdapterLayerArm64:20
        - "arn:aws:lambda:us-east-1:451483290750:layer:NewRelicPython310ARM64:27"
      Timeout: 15

or use the sample code provided by AWS itself and then apply the following environment variables:

AWS_LAMBDA_EXEC_WRAPPER: /opt/bootstrap
PORT: "8000"
ENV: !Ref StageName
NEW_RELIC_LAMBDA_HANDLER: run.sh
NEW_RELIC_ACCOUNT_ID: "<ACCOUNT_ID>"
NEW_RELIC_TRUSTED_ACCOUNT_KEY: "<ACCOUNT_ID>"
NEW_RELIC_LICENSE_KEY: "<API_KEY>"

Expected Behaviour

The value inside the environment variable NEW_RELIC_LAMBDA_HANDLER should be able to be located by the New Relic wrapper

Relevant Logs / Console output

Lambda log error: /opt/bootstrap: line 3: /var/task/newrelic_lambda_wrapper.handler: No such file or directory

Your Environment

• AWS Lambda
• Python 3.10 ARM runtime
• LambdaAdapterLayerArm64 layer
• NewRelicPython310ARM64 layer

Additional context

Is there any suggested workaround for this lambda scenario? I could not find anything in the docs and neither in any past issues from github

[workato-integration[bot]]

https://new-relic.atlassian.net/browse/NR-279309

[chaudharysaket]

Working on it

[chaudharysaket]

We can use the layerless approach which installs only the New Relic Extension that can transmit telemetry to New Relic. In the layerless approach, we don't need to set custom New Relic handler.

For lambda managed runtimes, need to investigate how to use the Agent+Extension layers as the AWS_LAMBDA_EXEC_WRAPPER is set to /opt/bootstrap while language agents also wrap the lambda handler.

[anentropic]

In general I find there's a lack of detail in the docs about what things actually do, so when the "happy path" instructions don't work I don't know where to go next...

the layerless docs show this:

import newrelic.agent
from newrelic_lambda.lambda_handler import lambda_handler

newrelic.agent.initialize()

@lambda_handler()
def handler(event, context):
  ...

it's unclear what the lambda_handler does and I am unsure how to integrate it in the Lambda Web Adapter case where I have a Django app

I am guessing I should ignore all the newrelic_lambda stuff and just treat it as a Django app that happens to run on Lambda

similarly I would love to know what NEW_RELIC_SERVERLESS_MODE_ENABLED does, to know whether I should really enable it or not in this set up