Error: loader.getUniqueId is not a function

[leVoT8]

🐛 Describe the bug

Model: GPT-4o Vector DB: Pinecone, serverless

import { RAGApplicationBuilder } from '@llm-tools/embedjs';
import { OpenAiEmbeddings } from '@llm-tools/embedjs-openai';
import { WebLoader } from '@llm-tools/embedjs-loader-web';
import { PineconeDb } from '@llm-tools/embedjs-pinecone'
import { OpenAi } from '@llm-tools/embedjs-openai'

//Replace this with your OpenAI key
process.env.OPENAI_API_KEY = "<xxx>"
process.env.PINECONE_API_KEY= "<xxx>"
const ragApplication = await new RAGApplicationBuilder()
.setModel(new OpenAi({model: "GPT-4o"}))
.setEmbeddingModel(new OpenAiEmbeddings())
.setVectorDb(new PineconeDb({
    projectName: 'medicalinfo',
    namespace: 'ns1',
    indexSpec: {
        serverless: {
            cloud: 'aws',
            environment: 'us-east-1'
        },
    },
}))
.build();

ragApplication.addLoader({ urlOrContent: 'https://www.forbes.com/profile/elon-musk' })
ragApplication.addLoader({ urlOrContent: 'https://en.wikipedia.org/wiki/Elon_Musk' })

await ragApplication.query('What is the net worth of Elon Musk today?')

Error:

.../node_modules/@llm-tools/embedjs/src/core/rag-application.js:134
        const uniqueId = loader.getUniqueId();
                                ^

TypeError: loader.getUniqueId is not a function
    at RAGApplication._addLoader (.../embedJS/node_modules/@llm-tools/embedjs/src/core/rag-application.js:134:33)
    at RAGApplication.addLoader (.../embedJS/node_modules/@llm-tools/embedjs/src/core/rag-application.js:121:21)
    at file:///.../embedJS/index.js:26:16
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v18.16.0

I am unsure if I did something wrong or if this is a bug.

Note: I exchanged { LanceDb } to { PineconeDb } on the import statement. { LanceDb } is what is written in the documentation, however, I think it was supposed to be { PineconeDb }

[adhityan]

The documentation needs fixing. Could you try replacing the addLoader line with -

await app.addLoader(new WebLoader({ urlOrContent: 'https://www.forbes.com/profile/elon-musk' }));

The library before versions 0.1.x used to support dynamic loaders - where you don't need to create an instance of a loader manually - this functionality was removed when the library moved into a monorepo - and will be available again later. The documentation wasn't updated properly.

[adhityan]

Find fixed / updated documentation here -

https://llm-tools.mintlify.app/components/vector-databases/pinecone