[Blocked] Unable to deploy Matano in a fresh AWS account - Lambda runtime deprecation for NodeJS 14.x

[IMG-PRCSNG]

Hello,

I tried to give matano a go on a fresh AWS account. But when I tried to deploy it, I get the following error message, saying AWS Lambda functions cannot be created with the NodeJS 14.x runtime anymore.

Failed resources:
MatanoDPCommonStack | 15:19:09 | CREATE_FAILED | AWS::Lambda::Function | 

DPCommonStack/Custom::S3AutoDeleteObjectsCustomResourceProvider/Handler (CustomS3AutoDeleteObjectsCustomResourceProviderHandler9D90184F) 

Resource handler returned message: "The runtime parameter of nodejs14.x is no longer supported for creating or updating AWS Lambda functions. 

We recommend you use the new runtime (nodejs20.x) while creating or updating functions. (Service: Lambda, Status Code: 400, HandlerErrorCode: InvalidRequest)

I checked the AWS Lambda documentation about runtimes and their support policies - it states that they have removed support for NodeJS 14 ended on Feb 8, 2024.

No new functions can be deployed / existing functions cant be updated with it.

[marklonghurst92]

Hi, we are also hitting the same issue, can someone please advise on how we resolve?

Blocking our deployments

Cheers

[timoguin]

The runtime version needs to be bumped here:

https://github.com/matanolabs/matano/blob/main/infra/lib/rust-function-layer/Dockerfile#L3

[rw0007]

Any idea when it will be fixed?

[IMG-PRCSNG]

Hey all,

Was able to get over this bump by upgrading aws-cdk to v2.85 and aws-sdk to something higher than 2.1311 that is compatible with v2.85. I also had to include a couple of files manually in the infra.pkg.json

There are newer version of CDK / SDK available - but I only managed to fix errors till 2.85. Newer versions throw more errors than what my puny little brain can handle :sweat_smile:

(2.85 is the lowest version of CDK with all the bug fixes that replaces the use of deprecated runtimes. See here and here )

The error is related to AWS deprecating the lambda runtimes - the old versions of CDK deploys Lambda function helpers with the deprecated runtimes, thus blocking all new deployments and updates, till we update the CDK dependency.

While this got me over the deployment problem, I am far from being able to tell if the new dependency versions would cause some runtime issues with Matano, due to hidden breaking changes / regression. Would need the help of maintainers to comment on the significance of sticking to AWS CDK v2.54 version.

This is the patch of the changes I applied aws-cdk.patch.txt

Pasting it here directly to make the changes easy to see.

Hope this helps!

diff --git a/cli/package.json b/cli/package.json
index 0100152..36f5da1 100644
--- a/cli/package.json
+++ b/cli/package.json
@@ -26,8 +26,8 @@
     "@oclif/plugin-help": "^5",
     "@oclif/plugin-plugins": "^2.0.1",
     "ansi-styles": "^5",
-    "aws-cdk": "2.54.0",
-    "aws-sdk": "^2.1093",
+    "aws-cdk": "2.85.0",
+    "aws-sdk": "^2.1311",
     "chalk": "^4",
     "enquirer": "^2.3",
     "execa": "^5",
@@ -110,4 +110,4 @@
   "prettier": {
     "printWidth": 120
   }
-}
+}
\ No newline at end of file
diff --git a/infra.pkg.json b/infra.pkg.json
index 892a58e..2562770 100644
--- a/infra.pkg.json
+++ b/infra.pkg.json
@@ -1,18 +1,25 @@
 {
   "name": "matano-cdk",
   "pkg": {
-    "scripts": ["./infra/dist/**/*.js"],
-    "targets": ["node14.18.1-linux-x64", "node14.18.1-macos-x64"],
+    "scripts": [
+      "./infra/dist/**/*.js"
+    ],
+    "targets": [
+      "node14.18.1-linux-x64",
+      "node14.18.1-macos-x64"
+    ],
     "assets": [
       "./data/**/*",
+      "./infra/node_modules/aws-cdk-lib/custom-resource-handlers/lib/aws-s3/auto-delete-objects-handler/*.js",
       "./infra/node_modules/aws-cdk-lib/custom-resources/lib/provider-framework/runtime/*.js",
       "./infra/node_modules/aws-cdk-lib/lambda-layer-awscli/**/*",
       "./infra/node_modules/aws-cdk-lib/asset-awscli-v1/**/*",
       "./infra/node_modules/@aws-cdk/asset-awscli-v1/**/*",
       "./infra/node_modules/aws-cdk-lib/aws-s3-assets/**/*",
       "./infra/node_modules/aws-cdk-lib/aws-s3-deployment/**/*",
-      "./infra/node_modules/aws-cdk-lib/aws-s3/**/*"
+      "./infra/node_modules/aws-cdk-lib/aws-s3/**/*",
+      "./infra/node_modules/aws-cdk-lib/lazy-index.js"
     ],
     "outputPath": "build"
   }
-}
+}
\ No newline at end of file
diff --git a/infra/package.json b/infra/package.json
index ffdc44c..4151995 100644
--- a/infra/package.json
+++ b/infra/package.json
@@ -24,9 +24,9 @@
   "dependencies": {
     "@aws-cdk/cx-api": "*",
     "@mrgrain/cdk-esbuild": "^3",
-    "aws-cdk": "2.54.0",
-    "aws-cdk-lib": "2.54.0",
-    "aws-sdk": "^2.1093",
+    "aws-cdk": "2.85.0",
+    "aws-cdk-lib": "2.85.0",
+    "aws-sdk": "^2.1311",
     "constructs": "^10.0.0",
     "fs-extra": "^10.1.0",
     "promptly": "^3.2.0",
@@ -39,4 +39,4 @@
   "prettier": {
     "printWidth": 120
   }
-}
+}
\ No newline at end of file
diff --git a/scripts/packaging/build.js b/scripts/packaging/build.js
index bde9c8f..5ef0144 100644
--- a/scripts/packaging/build.js
+++ b/scripts/packaging/build.js
@@ -43,7 +43,7 @@ function setCliPackageVersion() {
 }

 function prepareCdkPkg() {
-  execSync("npm install aws-cdk@2.54.0", { cwd: workDir });
+  execSync("npm install aws-cdk@2.85.0", { cwd: workDir });

   // some js template files mess up pkg, just delete
   fs.rmSync(path.resolve(workDir, "node_modules/aws-cdk/lib/init-templates/app/javascript"), {

[jlgore]

The above changes fixed my problems! Thank you.

[Stuxend]

Hi there, something to add here too, needs to be updated the python requeriments for the matano_detection lambda, i face a few error with my detections and i fix it using the solution posted here ++ adding this to the req>

aiobotocore==2.8.0

[Samrose-Ahmed]

I've deployed a change that should fix these issues.