AWS Lambda cannot open shared object file

[azatTemirbek]

Tried this combinations

• Layer: tried docker container | mac | ubuntu mint | cloud9 (v20 node) for building
• Runtimes: tried node v16|v18|v20 on lambda
• Logged file with readdirSync result is ['better_sqlite3.node']

Extra info

• Lambda uses EFS on /mnt/efs

Sample output when function invoked:

Response
{
  "errorType": "Error",
  "errorMessage": "/opt/nodejs/node20/node_modules/better-sqlite3/build/Release/better_sqlite3.node: cannot open shared object file: No such file or directory",
  "trace": [
    "Error: /opt/nodejs/node20/node_modules/better-sqlite3/build/Release/better_sqlite3.node: cannot open shared object file: No such file or directory",
    "    at Module._extensions..node (node:internal/modules/cjs/loader:1454:18)",
    "    at Module.load (node:internal/modules/cjs/loader:1208:32)",
    "    at Module._load (node:internal/modules/cjs/loader:1024:12)",
    "    at Module.require (node:internal/modules/cjs/loader:1233:19)",
    "    at require (node:internal/modules/helpers:179:18)",
    "    at bindings (/opt/nodejs/node20/node_modules/bindings/bindings.js:112:48)",
    "    at new Database (/opt/nodejs/node20/node_modules/better-sqlite3/lib/database.js:48:64)",
    "    at Object.<anonymous> (/private/var/folders/45/k19vcwvx031fqym53vzthzyw0000gn/T/tmpvmlaw9vw/src/search/file.ts:24:12)",
    "    at Module._compile (node:internal/modules/cjs/loader:1358:14)",
    "    at Module._extensions..js (node:internal/modules/cjs/loader:1416:10)"
  ]
}

[neoxpert]

Based on this blog post I would assume that the prebuilt Linux libraries are not compatible with Amazon Linux. So for now, you would have to setup a suitable build environment and compile better-sqlite3 for Amazon Linux on your own.

[azatTemirbek]

so basically when I am building layer I am using --use-container which is using public.ecr.aws/sam/build-nodejs18.x:latest-x86_64 Docker container image.

...
  SqliteLayer:
    Type: AWS::Serverless::LayerVersion
    Metadata:
      BuildMethod: makefile
    Properties:
      LayerName: "trakr-sqlite"
      Description: Runtime dependencies for Lambdas
      ContentUri: ./
      CompatibleRuntimes:
        - nodejs18.x
      CompatibleArchitectures:
        - arm64
      RetentionPolicy: Delete

.PHONY: build-SqliteLayer
build-SqliteLayer:
    mkdir -p "$(ARTIFACTS_DIR)/nodejs"
    cp package-sqlite.json "$(ARTIFACTS_DIR)/nodejs/package.json"
    npm install --production --legacy-peer-deps --prefix "$(ARTIFACTS_DIR)/nodejs/"
    rm "$(ARTIFACTS_DIR)/nodejs/package.json"
.PHONY: pack
pack:
    cd .aws-sam/build/SqliteLayer && zip -r trakr-sqlite-layer-v1.0.zip . && cp trakr-sqlite-layer-v1.0.zip ../../../

means I am building inside AWS container

sam build --use-container
Starting Build inside a container                                                                                                                                                                                                                  
Building layer 'SqliteLayer'                                                                                                                                                                                                                       
For container layer build, first compatible runtime is chosen as build target for container.                                                                                                                                                       

Fetching public.ecr.aws/sam/build-nodejs18.x:latest-x86_64 Docker container image......
Mounting ./trakr-layers as /tmp/samcli/source:ro,delegated, inside runtime container      

and in the source code I see better-sqlite3 uses prebuild-install and bindings lib to build release when the packege is installed

...
 "scripts": {
    "install": "prebuild-install || node-gyp rebuild --release", // <=== building release on install
    "build-release": "node-gyp rebuild --release",
    "build-debug": "node-gyp rebuild --debug",
    "rebuild-release": "npm run lzz && npm run build-release",
    "rebuild-debug": "npm run lzz && npm run build-debug",
    "test": "mocha --exit --slow=75 --timeout=5000",
    "benchmark": "node benchmark",
    "download": "bash ./deps/download.sh",
    "lzz": "lzz -hx hpp -sx cpp -k BETTER_SQLITE3 -d -hl -sl -e ./src/better_sqlite3.lzz"
  },
....

as a result I see the on the npm there is no build folder and inside layer it is exist which means native modules is build upon installation which is happening inside aws container (also tried inside cloud9 with AL 2023 image) error is same

[azatTemirbek]

ok I see prebuild-install is downloading from github but even after building it inside cloud9 and replacing the binary with my own binary I got same error

1. copied package.json to cloud9
2. npm install
3. cd node_modules/better-sqlite3
4. npm install --ignore-scripts
5. npx --no-install prebuild -r node -t 20.0.0 --include-regex 'better_sqlite3.node$' will build new build/Release/better_sqlite3.node
6. also run chmod 777 on better_sqlite3.node
7. packed to aws required structure

same same same error

[neoxpert]

And how does your Node project look like? In order to ensure that really your self compiled binary is loaded, I would ship it right next to your script files and pass it's path into the Database constructor using the nativeBinding option. Because otherwise, running install on your project might just overwrite the default binary location again.

[azatTemirbek]

@neoxpert after 2 days of digging seems I have been using arch incorrectly. Turns out after changing it to x86_64 on lambda config it is working ok.