Can't have embedded doc for a chat conversation

[ali-habibzadeh]

I am trying to have a conversation with my documents like this

const index = client.Index(indexName); //pinecode index
const queryEmbedding = await new OpenAIEmbeddings().embedQuery(question);
const queryResponse = await index.query({
  queryRequest: { topK: 10, vector: queryEmbedding, includeMetadata: true },
});
const llm = new OpenAI();
const chatHistory = new ChatMessageHistory();
const memory = new BufferMemory({ chatHistory, inputKey: "my_chat_history", memoryKey: "chat_history" });
const chain = new ConversationChain({ llm, memory, verbose: true });
const concatenatedPageContent = queryResponse.matches.map(match => (<any>match?.metadata)?.pageContent).join(" ");
const result = await chain.call({
  input_documents: [new Document({ pageContent: concatenatedPageContent })],
  input: question,
});

But gettiing

Error: Missing value for input history

Is there a way to give ConversationChain the ability to use input_documents from pinecone?

[jacoblee93]

The default prompt for ConversationChain requires memoryKey set to history

Also, you'll need to serialize the document to a string - the Document object will not work in that case.

In general, you may have better luck checking out/tweaking the following:

https://js.langchain.com/docs/guides/expression_language/cookbook#conversational-retrieval-chain

[ali-habibzadeh]

I managed to make the memory work but no way of making it understand input_documents

class LLMStuff {
  constructor(private indexName: string) {}
  private chain!: ConversationalRetrievalQAChain;
  private vectorStore!: PineconeStore;

  public async init() {
    const pinecone = new PineconeClient();
    await pinecone.init({
      apiKey: <string>process.env.PINECONE_API_KEY,
      environment: <string>process.env.PINECONE_ENVIRONMENT,
    });
    await this.setChain(pinecone, this.indexName);
  }

  private async setChain(client: PineconeClient, indexName: string) {
    const pineconeIndex = client.Index(indexName);
    this.vectorStore = await PineconeStore.fromExistingIndex(new OpenAIEmbeddings(), { pineconeIndex });
    const model = new OpenAI({ modelName: "gpt-4" });
    const memory = new BufferMemory({
      memoryKey: "chat_history",
      inputKey: "question",
      outputKey: "text",
      returnMessages: true,
    });
    this.chain = ConversationalRetrievalQAChain.fromLLM(model, this.vectorStore.asRetriever(), {
      returnSourceDocuments: true,
      memory,
    });
    console.log("initialised");
  }

  public async query(query: string): Promise<string> {
    const docs = await this.vectorStore.similaritySearch(query);
    const { text } = await this.chain.call({
      input_documents: [new Document({ pageContent: docs.map(doc => (<any>doc?.metadata)?.pageContent).join(" ") })],
      question: query,
    });
    return text;
  }
}

It seems if one wants to embed and use specific documents from vector then we have to use loadQAStuffChain which doesn't support conversation and if you ConversationalRetrievalQAChain with memory to have conversation then input_documents are ignored.

Here https://github.com/langchain-ai/langchain/discussions/1626 it seems someone managed to use loadQAChain of type stuff and pass memory however that seems impossible in typescript.

The below works fine for adding input_documents so the conversation is about that and not all the other stuff however no way to give it a memory to have a chat:

class LLMStuff1 {
  constructor(private indexName: string) {}
  private chain!: StuffDocumentsChain;
  private vectorStore!: PineconeStore;

  public async init() {
    const pinecone = new PineconeClient();
    await pinecone.init({
      apiKey: <string>process.env.PINECONE_API_KEY,
      environment: <string>process.env.PINECONE_ENVIRONMENT,
    });
    await this.setChain(pinecone, this.indexName);
  }

  private async setChain(client: PineconeClient, indexName: string) {
    const pineconeIndex = client.Index(indexName);
    this.vectorStore = await PineconeStore.fromExistingIndex(new OpenAIEmbeddings(), { pineconeIndex });
    const model = new OpenAI({ modelName: "gpt-4" });
    this.chain = loadQAStuffChain(model);
    console.log("initialised");
  }

  public async query(query: string): Promise<string> {
    const docs = await this.vectorStore.similaritySearch(query, 10);
    const { text } = await this.chain.call({
      input_documents: [new Document({ pageContent: docs.map(doc => (<any>doc?.metadata)?.pageContent).join(" ") })],
      question: query,
    });
    return text;
  }
}

[jacoblee93]

Why don't you try passing in docs directly? Not sure what that odd map is doing?

    const { text } = await this.chain.call({
      input_documents: docs,
      question: query,
    });

In general, chains should only have string arguments (the stuff QA chain is a special case).

[ali-habibzadeh]

Both codes use the exact same map. The difference is only the chai method used and conversational chain doesn't support input documents at all.