Duplicate middleware name 'NewRelicHeader'

[brunopenso]

Hello everyone,

I'm trying to run a lambda in AWS using this layer and I'm getting this error: Duplicate middleware name 'NewRelicHeader'

I'm trying to get the stacktrace but no success yet because of internal policies I don't have write access to cloud watch logs and all of our logs are sent to NewRelic, and there isn't any logs in NewRelic.

I'm using node 16 with serverless v3.

Here is my package.json(simplified):

{
  "scripts": {
    "start": "dotenv -- bash -c 'serverless offline --noPrependStageInUrl'",
  },
  "devDependencies": {
    "@types/jest": "^29.2.3",
    "coveralls": "^3.1.1",
    "dotenv": "^16.0.2",
    "dotenv-cli": "^6.0.0",
    "husky": "^8.0.2",
    "jest": "^29.3.1",
    "prettier": "^2.8.0",
    "serverless": "^3.25.0",
    "serverless-domain-manager": "^6.2.0",
    "serverless-newrelic-lambda-layers": "^3.3.7",
    "serverless-offline": "^11.5.0",
    "serverless-plugin-lambda-insights": "^1.5.0",
    "serverless-plugin-resource-tagging": "^1.2.0",
    "serverless-prune-plugin": "^2.0.1",
    "standard": "^17.0.0"
  },
  "dependencies": {
    "@google-cloud/pubsub": "^3.2.1",
    "@newrelic/winston-enricher": "^4.0.0",
    "newrelic": "^9.7.0",
    "serverless-plugin-log-retention": "^2.0.0",
    "webpack": "^5.75.0",
    "winston": "^3.8.2"
  },
  "engines": {
    "node": "16",
    "npm": "8"
  }
}

Here is the serverless.yml (simplified):

useDotenv: true
configValidationMode: warn

plugins:
  - serverless-prune-plugin
  - serverless-plugin-lambda-insights
  - serverless-domain-manager
  - serverless-newrelic-lambda-layers
  - serverless-offline

provider:
  name: aws
  architecture: arm64
  memorySize: ${env:MEMORYSIZE}
  region: ${env:AWS_REGION}
  runtime: nodejs16.x
  timeout: ${env:TIMEOUT}

  versionFunctions: true
  stackName: serverless-${self:service}-${env:ENVIRONMENT}
  tracing:
    apiGateway: true

  stackTags:
    Tier: backend

  logs:
    restApi:
      accessLogging: false
      executionLogging: true

package:
  individually: true
  excludeDevDependencies: true

custom:
  newRelic:
    accountId: ${env:NEWRELIC_ACCOUNT_ID}
    apiKey: ${env:NEWRELIC_API_KEY}
    disableAutoSubscription: true
    linkedAccount: ${env:VS}-${env:ENVIRONMENT}
    logLevel: ${env:NEWRELIC_LOG_LEVEL}
    trustedAccountKey: ${env:NEWRELIC_TRUSTED_ACCOUNT_KEY}

functions:
  isAlive:
    description: "due to bug/incompatibility between new relic/insights plugin for serverless, this function MUST NOT contain architecture attribute. Keep it on template until a hotfix is published."
    handler: "src/isAlive.isAlive"
  default:
    handler: handler.process
    architecture: ${self:provider.architecture} # must contain this attribute due to insights plugin.
    disableLogs: true
    timeout: 900
    events:
      - http:
          path: /default
          method: post

[workato-integration[bot]]

https://issues.newrelic.com/browse/NEWRELIC-5542

[brunopenso]

Some updates:

• I removed serverless-plugin-log-retention
• I changed the order of the layers to

  plugins:
  - serverless-prune-plugin
  - serverless-plugin-lambda-insights
  - serverless-domain-manager
  - serverless-offline
  - serverless-newrelic-lambda-layers

The error continue :(

[mrickard]

@brunopenso I see that you're including newrelic in your deployment package (as dependencies). That's the source of your trouble: you're installing that in your /var/task/node_modules...but since you're using the Lambda Layer (via this plugin), there's another agent that's also installed at /opt/nodejs/node_modules.

I'd assume that you've included New Relic explicitly because you're recording custom attributes, or doing something else that requires newrelic explicitly. What you can do is move it to devDependencies so that your function can use the locally-installed agent, without it being deployed also alongside your function. Then you can continue to use the layer-installed agent for instrumentation.

(NewRelicHeader is defined in the New Relic Node Agent, so if you have two dependencies attempting to define it, that suggests a duplicate agent. The proof of this is that you have an explicit dependency on the agent in addition to using the layer.)

[brunopenso]

HI @mrickard, thanks for the answer.

In the end the problem were related to the enricher + winston dependencies.

For the future users that arrive here:

You don't need to use newrelic and newrelic/enricher with lambda, because the layer is already doing the job for you :)