search

langchain-ai / langchainjs

🦜🔗 Build context-aware reasoning applications 🦜🔗
https://js.langchain.com/docs/
MIT License
12.71k stars 2.19k forks source link

OllamaEmbeddings throws vector size error in ElasticVectorSearch #6041

Closed rafheros closed 1 month ago

rafheros commented 4 months ago

Checked other resources

Example Code

 const loader = new PDFLoader("./sample-docs/embedded-images-tables.pdf");

  const docs = await loader.load();

  const textSplitter = new RecursiveCharacterTextSplitter({
    chunkSize: 500,
    chunkOverlap: 200,
  });

  const splits = await textSplitter.splitDocuments(docs);

  const model = new ChatOllama({
    model: 'mistral',
    temperature: 0,
    baseUrl: 'http://localhost:11434',
    useMMap: true,
  });

  const embeddings = new OllamaEmbeddings({model:"mxbai-embed-large", baseUrl: 'http://localhost:11434', onFailedAttempt: e => console.log(e), requestOptions: {
    useMMap: false,
    numCtx: 768,
  }});

  const vectorstore = await ElasticVectorSearch.fromDocuments(splits, embeddings, clientArgs);

Error Message and Stack Trace (if applicable)

⨯ Error: Failed to insert documents: [1:15305] failed to parse: The [dense_vector] field [embedding] in doc [document with id '7a62e1e2-af73-4f6b-a5c2-80556c56bd8f'] has more dimensions than defined in the mapping [768] [1:15346] failed to parse: The [dense_vector] field [embedding] in doc [document with id '11591cd4-9465-44ef-bae3-d347f16333b9'] has more dimensions than defined in the mapping [768] [1:15337] failed to parse: The [dense_vector] field [embedding] in doc [document with id '67725eab-b97f-41ee-9358-08f458d359c8'] has more dimensions than defined in the mapping [768] [1:15328] failed to parse: The [dense_vector] field [embedding] in doc [document with id 'c8cc5376-c88f-4215-ab52-46d4da890fad'] has more dimensions than defined in the mapping [768] [1:15296] failed to parse: The [dense_vector] field [embedding] in doc [document with id '6554fbbc-0793-4fff-a2c3-42c61ed7d513'] has more dimensions than defined in the mapping [768] at ElasticVectorSearch.addVectors (webpack-internal:///(rsc)/./nodemodules/.pnpm/@langchain+community@0.2.18@elastic+elasticsearch@8.14.0_@qdrant+js-client-rest@1.10.0_types_cr3nmz3isjncf76hurm3i4aloi/node_modules/@langchain/community/dist/vectorstores/elasticsearch.js:115:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async ElasticVectorSearch.fromDocuments (webpack-internal:///(rsc)/./nodemodules/.pnpm/@langchain+community@0.2.18@elastic+elasticsearch@8.14.0_@qdrant+js-client-rest@1.10.0_types_cr3nmz3isjncf76hurm3i4aloi/node_modules/@langchain/community/dist/vectorstores/elasticsearch.js:193:9) at async POST (webpack-internal:///(rsc)/./app/api/chat/route.ts:86:25) at async /home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:55038 at async ek.execute (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:45808) at async ek.handle (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:56292) at async doRender (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:1377:42) at async cacheEntry.responseCache.get.routeKind (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:1599:28) at async DevServer.renderToResponseWithComponentsImpl (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:1507:28) at async DevServer.renderPageComponent (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:1931:24) at async DevServer.renderToResponseImpl (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:1969:32) at async DevServer.pipeImpl (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:920:25) at async NextNodeServer.handleCatchallRenderRequest (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/next-server.js:272:17) at async DevServer.handleRequestImpl (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/base-server.js:816:17) at async /home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/dev/next-dev-server.js:339:20 at async Span.traceAsyncFn (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/trace/trace.js:154:20) at async DevServer.handleRequest (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/dev/next-dev-server.js:336:24) at async invokeRender (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/lib/router-server.js:174:21) at async handleRequest (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/lib/router-server.js:353:24) at async requestHandlerImpl (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/lib/router-server.js:377:13) at async Server.requestListener (/home/rafheros/next-langchain-app/node_modules/.pnpm/next@14.2.5_react-dom@18.3.1_react@18.3.1react@18.3.1/node_modules/next/dist/server/lib/start-server.js:141:13)

Description

I'm trying to load PDF files split them and store the results on a vector store running on a remote VM, I was unable to save it by any means using OllamaEmbedding because it throws the error above. It seams to be something related to vector dimensions.

I've also tried other Vector Stores like Chroma (throws 400: Bad Request) and Qdrant (also vector size error)

System Info

langchain 0.2.0 wsl/ubuntu (windows) Node v20.14.0 pnpm 9.5.0

dosubot[bot] commented 4 months ago

To resolve the vector size error when using OllamaEmbeddings with ElasticVectorSearch, you need to ensure that the vector size generated by the OllamaEmbeddings matches the expected vector size in your ElasticSearch index. Here are the steps to address this issue:

  1. Check the Model's Output Vector Size: Ensure that the model specified in OllamaEmbeddings (e.g., "mxbai-embed-large") produces vectors of the correct size. You may need to refer to the model's documentation or test the output size by embedding a sample document.

  2. Verify ElasticSearch Index Mapping: Ensure that the ElasticSearch index is configured to accept vectors of the size produced by the OllamaEmbeddings model. You can check the index mapping in ElasticSearch to confirm the expected vector size.

  3. Adjust Model or Index Configuration: If there is a mismatch, you may need to either change the model to one that produces the correct vector size or adjust the ElasticSearch index mapping to accept the vector size produced by your current model.

Here is an example of how you can embed documents using OllamaEmbeddings and ensure the index is correctly configured:

import { OllamaEmbeddings } from '@langchain/community/embeddings/ollama';
import { Client } from '@elastic/elasticsearch';
import { ElasticVectorSearch } from '@langchain/community/vectorstores/elasticsearch';

// Initialize ElasticSearch client
const client = new Client({ node: 'http://localhost:9200' });

// Initialize OllamaEmbeddings with the desired model
const ollamaEmbeddings = new OllamaEmbeddings({ model: 'mxbai-embed-large', baseUrl: 'http://localhost:11434' });

// Initialize ElasticVectorSearch
const elasticVectorSearch = new ElasticVectorSearch(ollamaEmbeddings, {
  client,
  indexName: 'documents',
  vectorSearchOptions: {
    engine: 'hnsw',
    similarity: 'l2_norm',
    m: 16,
    efConstruction: 100,
    candidates: 200,
  },
});

// Embed documents
const documents = ['Document 1 text', 'Document 2 text'];
ollamaEmbeddings.embedDocuments(documents).then(async (embeddings) => {
  console.log(embeddings);
  // Save embeddings to ElasticSearch
  // Ensure the vector size matches the ElasticSearch index mapping
  await elasticVectorSearch.addVectors(embeddings, documents);
});

If you continue to face issues, you may need to debug by logging the size of the embeddings generated and comparing it with the expected size in your ElasticSearch index. The ElasticVectorSearch class has methods to ensure the index exists and is correctly configured, which can help in resolving any mismatches [1][2].

To continue talking to Dosu, mention @dosu.