Langchain doesn't work on AWS Lambda with either faiss or hnswlib

[dmb0058]

The current version is broken on AWS lambda, both with Faiss and HNSWlib. I've spent the last few days building test cases on both platforms, but haven't dived into the langchain code.

The typical outcome is:

"Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory"

Same with HNSWlib (substitute hnswlib for faiss in the error above).

Also see

https://github.com/hwchase17/langchainjs/issues/943 https://github.com/hwchase17/langchainjs/issues/1764

Hope this helps - langchain is amazing, and if it worked on AWS it would be even better :)

David

[dosubot[bot]]

Answer generated by a 🤖

Answer

I understand that you're experiencing issues with LangChain on AWS Lambda, specifically when using either the faiss-node or hnswlib-node libraries. The error message you're encountering suggests that the AWS Lambda environment is missing a required shared library, libgomp.so.1, which is part of the GCC libraries.

Here's a potential solution that involves creating a Lambda Layer with the missing shared libraries and adding it to your AWS Lambda function:

1. On your local machine, create a directory, for instance, lambda-layer.
2. Within this directory, create another one named lib.
3. You'll need to install the necessary libraries in this directory. This can be done by running a Docker container that uses the Amazon Linux image, installing the libraries there, and then copying them to your local machine. Here's how you can do it:

docker run -v $(pwd):/lambda-layer -it amazonlinux:2
yum install -y gcc
cp /usr/lib64/libgomp.so.1 /lambda-layer/lib/
exit

4. Next, create a zip file of the lib directory:

cd lambda-layer
zip -r lib.zip lib

5. Upload this zip file as a new Lambda Layer in the AWS Lambda console.
6. Finally, add this layer to your Lambda function.

This approach should provide your function with the necessary shared libraries to run faiss-node and hnswlib-node.

Please note that this is one potential solution and there might be other ways to address this issue. If you find this solution to be correct and believe that it's a bug that could impact other users, we encourage you to make a pull request.

If this solution doesn't align with your experience, it might be worth checking if you're running the latest version of LangChain, as updates often include bug fixes and improvements.

Sources

• langchain/src/vectorstores/hnswlib.ts

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[dmb0058]

I think this might be a work-around, though it fell apart at

cp /usr/lib64/libgomp.so.1 /lambda-layer/lib/

I was lost in a sub-shell, in a directory I haven’t been able to debug yet, but I’ll have a look and see if I can tease the hints in the posts apart.

Thanks for your help :)

David

On 11 Jul 2023, at 18:39, dosu-beta[bot] @.***> wrote:

cp /usr/lib64/libgomp.so.1 /lambda-layer/lib/

[dmb0058]

Hi,

I successfully built a libgomp layer and attached it to my lambda (I don’t know if I did this correctly, as I selected Node 18.x as the runtime?)

Unfortunately the end result is very similar: looks like libm.so.6 is also “missing”? This whole thing smells of hackery and “nowhere ready for production” :)

EDIT: I checked for libm.so.6 in the Docker container but 2.26 is installed so I don't think this is really going anywhere. I guess it's a case of waiting until langchainjs and AWS Lambda work together, or just giving up on langchain as it's just as broken with hnswlib, for I guess the same reason.

Thanks anyway,

David

{ "errorType": "Error", "errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /opt/nodejs/node_modules/faiss-node/build/Release/faiss-node.node)", "trace": [ "Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g.npm install -S faiss-node.", "", "Error: /lib64/libm.so.6: versionGLIBC_2.27' not found (required by /opt/nodejs/node_modules/faiss-node/build/Release/faiss-node.node)", " at FaissStore.importFaiss (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:236:19)", " at async readIndex (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:141:37)", " at async Promise.all (index 1)", " at async FaissStore.load (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:144:51)", " at async query (/var/task/index.js:112:20)", " at async exports.handler (/var/task/index.js:20:9)" ] }

On 11 Jul 2023, at 18:39, dosu-beta[bot] @.***> wrote:

zip -r lib.zip lib

[jacoblee93]

Pinging @ewfian - is this something fixable in faiss-node?

[ewfian]

@jacoblee93 There is also a related issue on faiss-node side. I'll do some research and try to find a solution.

[dmb0058]

Happy to test any new versions if it helps :)

[lika85456]

There is a document on how to run hnswlib-node on AWS lambda. I managed to deploy it using this dockerfile:

FROM public.ecr.aws/lambda/nodejs:18 as builder
WORKDIR /usr/app

# install python
RUN yum install -y python3 make gcc-c++ && \
    yum clean all && \
    rm -rf /var/cache/yum

# install pnpm
RUN npm i -g pnpm

# install dependencies
COPY package.json pnpm-lock.yaml ./
RUN pnpm install

# build
COPY index.ts ./
RUN pnpm run build

FROM public.ecr.aws/lambda/nodejs:18
WORKDIR ${LAMBDA_TASK_ROOT}

COPY foo.dat ./
COPY --from=builder /usr/app ./
COPY --from=builder /usr/app/dist/* ./

CMD ["index.handler"]

Looking for a lambda layer solution, since I don't want to deploy using Docker images.

[dmb0058]

Amazon linux 2 image seems to only include /lib64/libm.so.6: version `GLIBC_2.26'

Any idea where I might find 2.27 just to see if adding it to my missing libraries layer would work? If not I'll spin up an EC2 server and build one.

[dmb0058]

... { "errorType": "Error", "errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /opt/nodejs/node_modules/faiss-node/build/Release/faiss-node.node)", "trace": [ "Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g.npm install -S faiss-node.", "", "Error: /lib64/libm.so.6: versionGLIBC_2.27' not found (required by /opt/nodejs/node_modules/faiss-node/build/Release/faiss-node.node)", " at FaissStore.importFaiss (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:236:19)", " at async readIndex (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:141:37)", " at async Promise.all (index 1)", " at async FaissStore.load (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:144:51)", " at async query (/var/task/index.js:112:20)", " at async exports.handler (/var/task/index.js:20:9)" ] } …

I think the problem here might be something to do with the fact that AWS Lambda runs a distro called "Amazon Linux 2", which has glibc 2.26. So in the error above it seems to be saying that faiss-node requires glibc 2.27. If that's the case it might explain why there are unmet dependencies. Unfortunately the only options would appear to be to wait for AWS to update the version of glibc or to get a 'downgraded' version of faiss-node for Lambda.

Or I might be barking at the moon :)

[campoblanco]

I'm facing the same issue, and haven't been able to solve it. I would really like to be able to use FAISS and LangChain in a lambda

[awadhwanan]

There is a document on how to run hnswlib-node on AWS lambda. I managed to deploy it using this dockerfile:

FROM public.ecr.aws/lambda/nodejs:18 as builder
WORKDIR /usr/app

# install python
RUN yum install -y python3 make gcc-c++ && \
    yum clean all && \
    rm -rf /var/cache/yum

# install pnpm
RUN npm i -g pnpm

# install dependencies
COPY package.json pnpm-lock.yaml ./
RUN pnpm install

# build
COPY index.ts ./
RUN pnpm run build

FROM public.ecr.aws/lambda/nodejs:18
WORKDIR ${LAMBDA_TASK_ROOT}

COPY foo.dat ./
COPY --from=builder /usr/app ./
COPY --from=builder /usr/app/dist/* ./

CMD ["index.handler"]

Looking for a lambda layer solution, since I don't want to deploy using Docker images.

@lika85456 This solution fails if you try to do anything with vectorStore https://js.langchain.com/docs/modules/chains/popular/vector_db_qa

[ewfian]

There is some update on faiss-node side, you can try it out, contact me if any questions https://github.com/ewfian/faiss-node/issues/25#issuecomment-1645315198

[dmb0058]

Hi,

No luck my side I’m afraid:

{ "errorType": "Error", "errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory", "trace": [ "Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.", "", "Error: libgomp.so.1: cannot open shared object file: No such file or directory", " at FaissStore.importFaiss (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:237:19)", " at async readIndex (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:141:37)", " at async Promise.all (index 1)", " at async FaissStore.load (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:144:51)", " at async query (/var/task/index.js:112:20)", " at async exports.handler (/var/task/index.js:20:9)" ] }

David

On 21 Jul 2023, at 10:56, Fan @.***> wrote:

There is some update on faiss-node side, you can try it out, contact me if any questions ewfian/faiss-node#25 (comment) https://github.com/ewfian/faiss-node/issues/25#issuecomment-1645315198 — Reply to this email directly, view it on GitHub https://github.com/hwchase17/langchainjs/issues/1930#issuecomment-1645323367, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXCQOXAGXESQIR2C5AOWLXRJG43ANCNFSM6AAAAAA2GKOL2A. You are receiving this because you authored the thread.

[ewfian]

@dmb0058 I have made a demo for langchain. Please confirm whether it works properly by uploading this zip directly to AWS. langchain-faiss-aws-demo.zip It should correctly output the total count in the index, which is 1.

---

If you want your project to work, please remember that the files(node_modules/faiss-node/build/Release) in the zip to upload to AWS should replace with the files I provided here: https://github.com/ewfian/faiss-node/issues/25#issuecomment-1645315198

[iAmCodeHead]

facing similar issue.

[ewfian]

Here is a new demo Function: langchain-aws-demo.zip Layer: faiss-node-v0.2.2-aws-layer-x64.zip

[2441630833]

The current version is broken on AWS lambda, both with Faiss and HNSWlib. I've spent the last few days building test cases on both platforms, but haven't dived into the langchain code.

The typical outcome is:

"Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory"

Same with HNSWlib (substitute hnswlib for faiss in the error above).

Also see

943 #1764

Hope this helps - langchain is amazing, and if it worked on AWS it would be even better :)

David

Hi, I encountered this issue too. I suspect the issue arises from langchain depends on the third-party modules faiss-node or hnswlib, I solved this issue by:

1. Remove the faiss-node dependency from your package.json file.
2. Modify your Dockerfile to install the faiss-node dependency directly.

If you're using HNSWlib, simply replace faiss with hnswlib in your Dockerfile.

By handling the installation independently in the Dockerfile instead of through package.json, you might avoid the issue. I hope this helps!

[dosubot[bot]]

Hi, @dmb0058! I'm Dosu, and I'm helping the langchainjs team manage their backlog. I wanted to let you know that we are marking this issue as stale.

Based on my understanding, the current issue is related to importing faiss-node and HNSWlib on AWS Lambda, resulting in an error message about a missing shared object file. There have been ongoing discussions and potential workarounds in the comments, including creating a Lambda Layer with the missing shared libraries and adding it to the AWS Lambda function. Docker solutions and updates on the faiss-node side have also been mentioned.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the langchainjs repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the langchainjs community!

[dmb0058]

Hey, thanks for checking in. Let me check - It all works fine if I manually upload the package from git but I'd really like to use the npm package as that's how everything else in my pipeline works. Will come back tomorrow with a go/no go.

David

On Mon, 13 Nov 2023, 16:02 dosubot[bot], @.***> wrote:

Hi, @dmb0058 https://github.com/dmb0058! I'm Dosu, and I'm helping the langchainjs team manage their backlog. I wanted to let you know that we are marking this issue as stale.

Based on my understanding, the current issue is related to importing faiss-node and HNSWlib on AWS Lambda, resulting in an error message about a missing shared object file. There have been ongoing discussions and potential workarounds in the comments, including creating a Lambda Layer with the missing shared libraries and adding it to the AWS Lambda function. Docker solutions and updates on the faiss-node side have also been mentioned.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the langchainjs repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the langchainjs community!

— Reply to this email directly, view it on GitHub https://github.com/langchain-ai/langchainjs/issues/1930#issuecomment-1808455915, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXCQOOMSJPG2CSHAYJKNTYEJACPAVCNFSM6AAAAAA2GKOL2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBYGQ2TKOJRGU . You are receiving this because you were mentioned.Message ID: @.***>

[Julien-cpsn]

I am facing the same issue inside of Vue 3 + Vite

[ashishbamania]

I have been using Next.js 13 and was getting the same issue ("Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.") with the following code:

  const embeddings = new OpenAIEmbeddings({
    openAIApiKey: process.env.OPENAI_API_KEY,
  });

  const vectorStore = await FaissStore.fromDocuments(
    docs,
    embeddings,
  );

I fixed it with making these changes to next.config.js file:

/** @type {import('next').NextConfig} */
const nextConfig = {
  webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
    config.resolve.alias.canvas = false;
    config.resolve.alias.encoding = false;
    config.externals = [...config.externals, "faiss-node"]; //added this line
    return config;
  },
};

module.exports = nextConfig;

[jacoblee93]

Are you using the edge runtime? faiss-node is Node only :(

[bitsnaps]

I'm getting the same issue with faiss-node@^0.5.1 (also tried ^0.2.1 which was working with an old example).

...
Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node`.
...

Running locally with Node v18.19.0 LTS on macOS. I used HuggingFaceInferenceEmbeddings instead of openai's embeddings.

[dosubot[bot]]

🤖

Based on the error messages you've shared, it seems like the AWS Lambda environment is missing some shared libraries required by faiss-node and hnswlib. Specifically, the errors mention libgomp.so.1 and /lib64/libm.so.6: version GLIBC_2.27.

One possible workaround is to create a custom AWS Lambda layer that includes these missing shared libraries. Here's a general outline of the steps:

1. Create a Dockerfile that uses the Amazon Linux image (which is the same environment that AWS Lambda runs on). In this Dockerfile, install the necessary dependencies and build the shared object files.

2. Use the Dockerfile to build a Docker image and run a Docker container.

3. Inside the Docker container, create a zip file that contains the shared object files and dependencies.

4. Upload the zip file to AWS S3.

5. Create a new AWS Lambda layer and point it to the zip file in S3.

6. Add the new layer to your AWS Lambda function.

Please note that the exact steps and code will depend on the specific shared object files and dependencies that are missing.

In the context of LangChain, you would need to ensure that the AWS Lambda environment has the necessary dependencies to run Faiss or HNSWlib, and that the shared object files are included in the AWS Lambda layer.

Unfortunately, without more specific information about the shared object files and dependencies required by Faiss and HNSWlib, I can't provide more specific steps or code. You might need to consult the documentation or community for these libraries to get this information.

As for the issue with GLIBC_2.27, it seems like AWS Lambda's Amazon Linux 2 environment only includes GLIBC_2.26. This might be causing the unmet dependencies issue. One possible solution is to wait for AWS to update the version of glibc in their environment, or to find a version of faiss-node that is compatible with GLIBC_2.26.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

• examples/src/indexes/vector_stores/faiss_fromdocs.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[apotox]

here is an example to how to use / install faiss node on aws lambda https://github.com/apotox/faiss-node-aws-lambda

[dosubot[bot]]

Hi, @dmb0058,

I'm helping the langchainjs team manage their backlog and am marking this issue as stale. The issue #1930 highlighted the problem of Langchain being broken on AWS Lambda with both Faiss and HNSWlib, resulting in an error related to missing shared object files. There have been potential workarounds discussed in the comments, including creating a Lambda Layer with the missing shared libraries, Docker solutions, and updates on the faiss-node side. Users have also shared their experiences and attempts to address the issue, with some providing Dockerfile examples and AWS Lambda layer solutions.

Could you please let us know if this issue is still relevant to the latest version of the repository? If it is, please comment on the issue to let the langchainjs team know. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[dmb0058]

Hi,

I never did get it working, apart from the x86 version of faiss that Fan (ewfian) built ( https://github.com/ewfian/faiss-node/releases/download/v0.5.1/faiss-node-v0.5.1-aws-x64.zip) which I've used since then.

I couldn't get the equivalent arm64 one to load, and on the odd occasions where I tried to load newer x86 versions they didn't work either. It seems to be missing dependencies from what I can remember, but as far as I can tell this is the only FAISS that works as an AWS lambda layer with langchainjs.

I gave up completely on HNSWlib.

David

On Sun, 19 May 2024, 17:04 dosubot[bot], @.***> wrote:

Hi, @dmb0058 https://github.com/dmb0058,

I'm helping the langchainjs team manage their backlog and am marking this issue as stale. The issue #1930 https://github.com/langchain-ai/langchainjs/issues/1930 highlighted the problem of Langchain being broken on AWS Lambda with both Faiss and HNSWlib, resulting in an error related to missing shared object files. There have been potential workarounds discussed in the comments, including creating a Lambda Layer with the missing shared libraries, Docker solutions, and updates on the faiss-node side. Users have also shared their experiences and attempts to address the issue, with some providing Dockerfile examples and AWS Lambda layer solutions.

Could you please let us know if this issue is still relevant to the latest version of the repository? If it is, please comment on the issue to let the langchainjs team know. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

— Reply to this email directly, view it on GitHub https://github.com/langchain-ai/langchainjs/issues/1930#issuecomment-2119286685, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXCQM4B2UIMPQ3GCQAIVTZDDESLAVCNFSM6AAAAAA2GKOL2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZGI4DMNRYGU . You are receiving this because you were mentioned.Message ID: @.***>