Output values have 2 keys with initializeAgentExecutorWithOptions

[app-rc]

Hi ✋ ,

I am using the function initializeAgentExecutorWithOptions to set up an intelligent chatbot that can search among several [tools].

// Create a vector store from the documents.
const vectorStore = await QdrantVectorStore.fromExistingCollection(
    new OpenAIEmbeddings(embedding),
    { collectionName }
)

// Initialize a retriever wrapper around the vector store
const vectorStoreRetriever = vectorStore.asRetriever({ k: 1 })

const tool = createRetrieverTool(vectorStoreRetriever, {
    name: 'search_sales_book',
    description: 'search up to date informations about sales processes,
})

const reference = new DynamicStructuredTool({
    name: 'search_references',
    description: 'search into clients references',
    schema: z.object({
        location: z.object({
            lat: z.number().describe('Latitude'),
            lon: z.number().describe('Longitude'),
            radius: z.number().describe('Radius (km)'),
        }),
    }),
    func: async (input) => {
        try {
                const { location } = input

                if (!location) {
                    return 'Sorry no location provided'
                }

            const { lat, lon, radius } = location

            const filter = {
                must: [{
                    key: 'location',
                    geo_radius: {
                        center: {
                            lat,
                            lon,
                        },
                        radius: radius * 1000,
                    },
                }],
            }
            const results = await qdrant.scroll('references', { filter })

            if (results.points.length > 0) {
                return results.points.map(p => p.payload.content).join('\n\n')
            } else {
                return "Sorry, no data"
            }
        } catch (err) {
            return "Sorry, unavailable service"
        }
    },
    outputKey: 'output',
})

const model = new ChatOpenAI({
    temperature: 0,
    maxTokens: 300,
    streaming: true,
})

const chatHistory = new ChatMessageHistory(
    previousMessages.map(convertToLangChainMessages),
)

const memory = new OpenAIAgentTokenBufferMemory({
    llm: model,
    memoryKey: 'chat_history',
    chatHistory,
    outputKey: 'output',
})

const executor = await initializeAgentExecutorWithOptions([tool, reference], model, {
    agentType: 'openai-functions',
    memory,
    // verbose: true,
    outputKey: 'output',
    returnIntermediateSteps: true,
    agentArgs: {
        prefix: PREFIX,
    },
})

executor.call({
    input: currentMessageContent,
}, {
    callbacks: [handlers],
})

When it searches in multiple tools at the same request, it gives me an error message:

Error: output values have 2 keys, you must specify an output key or pass only 1 key as output
    at getOutputValue (webpack-internal:///(rsc)/./node_modules/langchain/dist/memory/base.js:46:15)
    at OpenAIAgentTokenBufferMemory.saveContext (webpack-internal:///(rsc)/./node_modules/langchain/dist/agents/toolkits/conversational_retrieval/token_buffer_memory.js:110:92)
    at AgentExecutor.call (webpack-internal:///(rsc)/./node_modules/langchain/dist/chains/base.js:124:31)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

I added some console.log into getOutputValue to check inputs

export const getOutputValue = (outputValues, outputKey) => {
    const value = getValue(outputValues, outputKey);
    if (!value) {
        const keys = Object.keys(outputValues);
        throw new Error(`output values have ${keys.length} keys, you must specify an output key or pass only 1 key as output`);
    }
    return value;
};

And get something like this

{
  outputValues: {
    output: '',
    intermediateSteps: [...],
  },
  outputKey: 'output'
}

If tried to remove returnIntermediateSteps as recommended here #2664 but fot this message error...

TypeError: Cannot read properties of undefined (reading 'flatMap')
    at _formatIntermediateSteps (webpack-internal:///(rsc)/./node_modules/langchain/dist/agents/openai/index.js:66:30)
    at OpenAIAgentTokenBufferMemory.saveContext (webpack-internal:///(rsc)/./node_modules/langchain/dist/agents/toolkits/conversational_retrieval/token_buffer_memory.js:112:116)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async AgentExecutor.call (webpack-internal:///(rsc)/./node_modules/langchain/dist/chains/base.js:124:13)

Do you have any ideas where this might be coming from? Thank you !

[dosubot[bot]]

🤖

Hello,

The error message you're receiving is likely due to the fact that the function initializeAgentExecutorWithOptions is expecting a single output key when multiple keys are being returned. This can happen if the agent you're using is designed to return multiple outputs.

In the context provided, the initializeAgentExecutorWithOptions function is used to initialize an AgentExecutor with a specific agent type and a set of tools. However, there's no explicit handling of the output keys in the provided code.

To resolve this issue, you need to specify the output key you're interested in when calling the executor.call function. For example, if your agent returns two keys, 'key1' and 'key2', and you're interested in 'key1', you should modify your call to something like this:

const result = await executor.call({ input, outputKey: 'key1' });

This way, the executor knows which key to use for the output. If you want to handle multiple keys, you'll need to call the executor separately for each key or modify your agent to return a single key.

Regarding the second error you're encountering when you remove returnIntermediateSteps, it seems like the _formatIntermediateSteps function is trying to access a property of an undefined object. This could be due to the fact that the intermediateSteps property is not being returned when returnIntermediateSteps is set to false. You might need to add a check in your code to ensure that intermediateSteps is defined before trying to access its properties.

I hope this helps! Let me know if you have any other questions.

Sources

• DynamoDBChatMessageHistory not working with initializeAgentExecutorWithOptions
• langchain/src/agents/tests/agent.int.test.ts
• langchain/src/agents/initialize.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Thinkscape]

Hitting the same problem. Whenever I have memory enabled I must have returnIntermediateSteps: true or it will fail with the flatMap error.

[dosubot[bot]]

Hi, @app-rc,

I'm helping the langchainjs team manage our backlog and am marking this issue as stale. From what I understand, you raised an issue regarding the output values having 2 keys when searching in multiple tools at the same request, resulting in an error message. I attempted to resolve the issue by removing returnIntermediateSteps as recommended, but encountered a new error. In the comments, I provided guidance on handling multiple output keys and addressing the second error related to removing returnIntermediateSteps. Another user, @Thinkscape, also reported encountering a similar problem when using memory and the necessity of having returnIntermediateSteps: true.

Could you please confirm if this issue is still relevant to the latest version of the langchainjs repository? If it is, please let the langchainjs team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you!