Failed on sqlite_vss.load(db) due to cannot open shared object file

[github-neo]

I can't run my code with sqlite-vss, could anyone help look into below issue? Thank you!

OS: WSL (Ubuntu 22.04) in Windows 10

When I run below code:

...
...
import Database from "better-sqlite3";
import * as sqlite_vss from "sqlite-vss";

const db = new Database("chat-doc.db");
sqlite_vss.load(db);
...
...

It always failed on the sqlite_vss.load(db); statement due to below error:

[nodemon] starting `node index.mjs`

/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/better-sqlite3/lib/methods/wrappers.js:19
        this[cppdb].loadExtension(...args);
                    ^
SqliteError: /mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0.so.so: cannot open shared object file: No such file or directory
    at Database.loadExtension (/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/better-sqlite3/lib/methods/wrappers.js:19:14)
    at loadVss (file:///mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss/src/index.js:65:6)
    at Module.load (file:///mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss/src/index.js:69:3)
    at file:///mnt/d/GitHub/doc-chat-poc/express-server/embeddings.mjs:14:12
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
  code: 'SQLITE_ERROR'
}

Node.js v18.17.1
[nodemon] app crashed - waiting for file changes before starting...

Looks like it always try to find "vss0.so.so" file, but in fact there is only "vss0.so" file in the path, there is no any *.so.so file.

Here is my package.json:

{
  "name": "express-server",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node index.mjs",
    "dev": "nodemon index.mjs"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "better-sqlite3": "^8.6.0",
    "dotenv": "^16.3.1",
    "express": "^4.18.2",
    "langchain": "^0.0.146",
    "sqlite-vss": "^0.1.2"
  },
  "devDependencies": {
    "@types/better-sqlite3": "^7.6.4",
    "@types/express": "^4.17.17",
    "@types/node": "^20.6.0",
    "nodemon": "^3.0.1",
    "typescript": "^5.2.2"
  }
}

[asg017]

Thanks for reporting!

Just curious, if you replaced that sqlite_vss.load(db) line with:

db.loadExtension('/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0');

or:

db.loadExtension('/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0.so');

Would either of those work?

[github-neo]

Thanks for reporting!

Just curious, if you replaced that sqlite_vss.load(db) line with:

db.loadExtension('/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0');

or:

db.loadExtension('/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0.so');

Would either of those work?

Here are the results:

db.loadExtension('/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0');:

/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/better-sqlite3/lib/methods/wrappers.js:19
        this[cppdb].loadExtension(...args);
                    ^
SqliteError: libblas.so.3: cannot open shared object file: No such file or directory
    at Database.loadExtension (/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/better-sqlite3/lib/methods/wrappers.js:19:14)
    at file:///mnt/d/GitHub/doc-chat-poc/express-server/embeddings.mjs:15:4
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
  code: 'SQLITE_ERROR'
}

db.loadExtension('/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0.so');:

/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/better-sqlite3/lib/methods/wrappers.js:19
        this[cppdb].loadExtension(...args);
                    ^
SqliteError: /mnt/d/GitHub/doc-chat-poc/express-server/node_modules/sqlite-vss-linux-x64/lib/vss0.so.so: cannot open shared object file: No such file or directory
    at Database.loadExtension (/mnt/d/GitHub/doc-chat-poc/express-server/node_modules/better-sqlite3/lib/methods/wrappers.js:19:14)
    at file:///mnt/d/GitHub/doc-chat-poc/express-server/embeddings.mjs:15:4
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
  code: 'SQLITE_ERROR'
}

[github-neo]

BTW, it failed too due to the same issue even when run in Docker.

My Dockerfile:

# Use the base image with Node.js 18
FROM node:18

# Create a working directory inside the container
WORKDIR /app

# Copy package.json and package-lock.json to the working directory
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy your application files to the working directory
COPY . .

# Expose the port used by the application
EXPOSE 3200

# Run your application when the container starts
CMD ["npm", "start"]

[github-neo]

Update: This issue has been resolved after I run sudo apt-get install -y libgomp1 libatlas-base-dev liblapack-dev! 😁 Sorry I missed this note in the readme. 🤣

[github-neo]

Update: This issue has been resolved after I run sudo apt-get install -y libgomp1 libatlas-base-dev liblapack-dev! 😁 Sorry I missed this note in the readme. 🤣

[asg017]

Glad to heat it got resolved!

Pretty weird though, I would expect a different error message if those packages weren't installed. Thanks for sharing, will look out for it in the future!