Error "Please install hnswlib-node as a dependency with, e.g. `npm install -S hnswlib-node`" on M2 Chip

[clemenspeters]

While the exact same docker image builds just fine on the linux AWS EC2 servers and in github actions, on my M2 Chip MacBook I get

#0 14.81 file:///app/node_modules/langchain/dist/vectorstores/hnswlib.js:172
#0 14.81             throw new Error("Please install hnswlib-node as a dependency with, e.g. `npm install -S hnswlib-node`");
#0 14.81                   ^
#0 14.81 
#0 14.81 Error: Please install hnswlib-node as a dependency with, e.g. `npm install -S hnswlib-node`
#0 14.81     at HNSWLib.imports (file:///app/node_modules/langchain/dist/vectorstores/hnswlib.js:172:19)
#0 14.81     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
#0 14.81     at async HNSWLib.getHierarchicalNSW (file:///app/node_modules/langchain/dist/vectorstores/hnswlib.js:35:37)
#0 14.81     at async HNSWLib.initIndex (file:///app/node_modules/langchain/dist/vectorstores/hnswlib.js:49:26)
#0 14.81     at async HNSWLib.addVectors (file:///app/node_modules/langchain/dist/vectorstores/hnswlib.js:68:9)
#0 14.81     at async HNSWLib.fromDocuments (file:///app/node_modules/langchain/dist/vectorstores/hnswlib.js:163:9)

when running HNSWLib.fromDocuments(docs, new OpenAIEmbeddings({...embeddingsOpts, openAIApiKey:OPENAI_API_KEY})); is called.

So it works on the server (luckily), but not on my local machine in docker 🤔 😕

Dockerfile:

FROM node:18-alpine AS BUILD_IMAGE

RUN apk update && \
    apk add -q make g++ python3
.
.
.

[mojalil]

I have the exact same issues and i'm on a mac m1 pro.

[duyhnguyen216]

Does not work on Azure :(

[JariHuomo]

Had this error on my Electron project, got it fixed it with import change:

static async imports() { try {

       const HierarchicalNSW = require('hnswlib-node').HierarchicalNSW;

        return { HierarchicalNSW };
    }
    catch (err) {
        throw new Error(err);
    }
}

[dmb0058]

Doesn't work on AWS Lambda either ... maybe a problem with langchainjs but see https://github.com/hwchase17/langchainjs/issues/1930

[kasem-sm]

Had this error on my Electron project, got it fixed it with import change:

static async imports() { try {

       const HierarchicalNSW = require('hnswlib-node').HierarchicalNSW;

        return { HierarchicalNSW };
    }
    catch (err) {
        throw new Error(err);
    }
}

Thanks, man! I was trying to solve this for the past 5 hours (on electron).

[Lookbid]

Having same issue

[HamzaElkotp]

Did anyone found a solution?

[dmb0058]

I had that problem on AWS Lambda with HNSWlib and Faiss due to missing dependencies. The Faiss developer fixed these so I switched Faiss in. However I can make that error appear if I attach the x86_64 Faiss layer to an arm64 based lambda. Possibly related, maybe not.

[Lukem121]

Think I figured it out, you need to install Visual Studio.

https://visualstudio.microsoft.com/downloads/

[alexomon018]

I had the same problem, but in the end I ended up using: import { MemoryVectorStore } from 'langchain/vectorstores/memory';

const embeddings = new OpenAIEmbeddings(); const store = await MemoryVectorStore.fromDocuments(docs, embeddings);

Also tried installing Visual Studio and it didn't helped :/

[shridhar-tl]

Hi @alexomon018, I'm facing same issue.

I am using it in GatsbyAPI and deploying it to Netlify. It doesn't work in local and in netlify as well.

I can try switching to MemoryVectorStore, but how can I serialize the data and use it for future again. I was able to do it in HNSWLib using save method.

[Update: I figured how to searilize MemoryVestorStore and use it. So I removed usage of HNSWLib and proceeded forward]

[Riyaancode]

I had the same problem, but in the end I ended up using: import { MemoryVectorStore } from 'langchain/vectorstores/memory';

const embeddings = new OpenAIEmbeddings(); const store = await MemoryVectorStore.fromDocuments(docs, embeddings);

Also tried installing Visual Studio and it didn't helped :/

@alexomon018 Thanks man it's worked for me

[cshaxu]

having exactly the same issue... is it that langchain wasn't able to import this lib correctly somehow?