ewfian / faiss-node

Node.js bindings for faiss
https://www.npmjs.com/package/faiss-node
MIT License
107 stars 10 forks source link

dlopen Symbol not found error on macOS 10.15.7 and Netlify #11

Open JavascriptMick opened 1 year ago

JavascriptMick commented 1 year ago

Describe the bug Trying to run the following code...

import { FaissStore } from "langchain/vectorstores/faiss";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
const vectorStore = await FaissStore.loadFromPython('./data/saved_index/', new OpenAIEmbeddings());
const results = await vectorStore.similaritySearch(query, 5);

Fails with the following Error on macOS 10.15.7

Uncaught (in promise) TRPCClientError: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node` and make sure you have `libomp` installed in your path.

Error: dlopen(/Users/michael/Workspace/potteryhelper.com/node_modules/faiss-node/build/Release/faiss-node.node, 1): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
  Referenced from: /Users/michael/Workspace/potteryhelper.com/node_modules/faiss-node/build/Release/faiss-node.node (which was built for Mac OS X 12.6)
  Expected in: /usr/lib/libc++.1.dylib

    at TRPCClientError.from (transformResult-6fb67924.mjs?v=5f4c615b:4:20)
    at httpBatchLink.mjs?v=5f4c615b:190:56
from @ transformResult-6fb67924.mjs?v=5f4c615b:4
(anonymous) @ httpBatchLink.mjs?v=5f4c615b:190
Promise.catch (async)
callWithAsyncErrorHandling @ runtime-core.esm-bundler.js:168
invoker @ runtime-dom.esm-bundler.js:345

and this error on Netlify...

May 29, 12:37:31 AM: 36d0dca3 INFO   Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node` and make sure you have `libomp` installed in your path.

Error: libgomp.so.1: cannot open shared object file: No such file or directory

Environment:

To Reproduce Steps to reproduce the behavior:

  1. npm install faiss-node and "langchain": "^0.0.84"
  2. run the above code

Expected behavior index is rebuilt from python files and returns data

Screenshots

Additional context I was able to get my local environment running by 1) git checkout faiss-node & npm install 2) Upgrade my CMake 3) npm run build 4) npm link 5) then back in my project... npm link faiss-node... effectively swapping out the locally built module for my project and it's dependencies

So I guess it's just an issue with compiling the library for older versions of macOS?

Here are my package.json dependencies

  "dependencies": {
    "@pinia/nuxt": "^0.4.6",
    "@trpc/client": "^10.9.0",
    "@trpc/server": "^10.9.0",
    "daisyui": "^2.51.5",
    "faiss-node": "^0.2.0",
    "generate-password-ts": "^1.6.3",
    "langchain": "^0.0.84",
    "openai": "^3.2.1",
    "pickleparser": "^0.1.0",
    "pinia": "^2.0.30",
    "stripe": "^11.12.0",
    "superjson": "^1.12.2",
    "trpc-nuxt": "^0.8.0",
    "zod": "^3.20.2"
  },

and faiss-node in my package-lock.json...

    "node_modules/faiss-node": {
      "version": "0.2.0",
      "resolved": "https://registry.npmjs.org/faiss-node/-/faiss-node-0.2.0.tgz",
      "integrity": "sha512-4HNyK0WX26OTb4UbaJh810wY6gCYag53NssjMKCA9GT37bP0reGXz6YGOBjpbC/EzqJlYvsqKcDIzbJSfkTieg==",
      "hasInstallScript": true,
      "dependencies": {
        "bindings": "^1.5.0",
        "node-addon-api": "^6.0.0",
        "prebuild-install": "^7.1.1"
      },
      "engines": {
        "node": ">= 14.0.0"
      }
    },
ewfian commented 1 year ago

@JavascriptMick Thanks for reporting the issue.

  1. For error on macOS 10.15.7

    So I guess it's just an issue with compiling the library for older versions of macOS?

I think this is the reason. The pre-build binary was compiled on macOS 12.6, some symbols on new version may not exist on the old version. Compiling it yourself may be a good option for now. I will add documentation later on how to compile.

  1. For error on Netlify

Please try to install the missing package to see if it helps sudo apt-get install libgomp1 or yum install libgomp


I will further study whether there is a better solution on this issue.