TypeError: Cannot read properties of undefined (reading 'additional_kwargs')

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

const { ChatMistralAI } = require( "@langchain/mistralai" );
var lc_core = require(  "@langchain/core/messages" );

if (req.body.stream) {

      const model = new ChatMistralAI({apiKey: MistralClientModule.apiKey,modelName: req.body.model, });
      console.log('lc_core: ', lc_core)
      const response = await model.stream(new lc_core.ChatMessage(req.body.messages.content, false));

      return response;
}

Error Message and Stack Trace (if applicable)

lc_core: {
BaseMessage: [class BaseMessage extends Serializable],
BaseMessageChunk: [class BaseMessageChunk extends BaseMessage],
HumanMessage: [class HumanMessage extends BaseMessage],
HumanMessageChunk: [class HumanMessageChunk extends BaseMessageChunk],
AIMessage: [class AIMessage extends BaseMessage],
AIMessageChunk: [class AIMessageChunk extends BaseMessageChunk],
SystemMessage: [class SystemMessage extends BaseMessage],
SystemMessageChunk: [class SystemMessageChunk extends BaseMessageChunk],
FunctionMessage: [class FunctionMessage extends BaseMessage],
FunctionMessageChunk: [class FunctionMessageChunk extends BaseMessageChunk],
ToolMessage: [class ToolMessage extends BaseMessage],
ToolMessageChunk: [class ToolMessageChunk extends BaseMessageChunk],
ChatMessage: [class ChatMessage extends BaseMessage],
isBaseMessage: [Function: isBaseMessage],
isBaseMessageChunk: [Function: isBaseMessageChunk],
coerceMessageLikeToMessage: [Function: coerceMessageLikeToMessage],
ChatMessageChunk: [class ChatMessageChunk extends BaseMessageChunk],
getBufferString: [Function: getBufferString],
mapStoredMessageToChatMessage: [Function: mapStoredMessageToChatMessage],
mapStoredMessagesToChatMessages: [Function: mapStoredMessagesToChatMessages],
mapChatMessagesToStoredMessages: [Function: mapChatMessagesToStoredMessages]
}
TypeError: Cannot read properties of undefined (reading 'additional_kwargs')
at new BaseMessage (/usr/src/app/function/node_modules/@langchain/core/dist/messages/index.cjs:57:21)
at new ChatMessage (/usr/src/app/function/node_modules/@langchain/core/dist/messages/index.cjs:441:9)
at mistralai_api (/usr/src/app/function/handler.js:68:53)

Description

Trying to use @langchain/mistralai as a wrapper over the @mistralai/mistralai nodejs client. Worth mentioning, the @mistralai/mistralai client is working fine.... I have tried to align the versions of @langchain/core using the caret notation in package.json; I've tried to use ChatMessage, HumanMessage and BaseMessage as well. Always the same error. Thank you

System Info

nodejs version 18

package.json

{ 
"name": "mistral-gagnant",
"version": "1.1.0",

"dependencies": {
"axios": "latest",
"qs":"latest",
"util": "latest",
"cookie": "latest", 
"btoa": "latest",         
"lodash": "latest",    
"@kubernetes/client-node": "latest",
"@mistralai/mistralai": "latest",
"@langchain/mistralai": "latest",
"@langchain/core": "latest"
},
"devDependencies": {
"cheerio": "latest"
}  
}

Hey @ptesny! Great to see you diving into another challenge. Let's see what we can do about this one.

Thank you for providing detailed information about the issue you're facing. The error TypeError: Cannot read properties of undefined (reading 'additional_kwargs') suggests that the object expected to have the additional_kwargs property is undefined. This issue might be arising from how the withStructuredOutput method is being used or how the response from the model is being handled.

Given the context and the code snippet, the issue might not be directly visible in the provided code. However, the error typically occurs when trying to access a property on an undefined object. In this case, it seems related to the handling of the response from the model, particularly when dealing with the includeRaw option.

To resolve this issue, ensure that every part of the response object where additional_kwargs is accessed is properly checked for undefined or null values. This can be done by optional chaining (?.) or by explicitly checking the object before accessing its properties.

Here's a revised approach to safely access additional_kwargs, applying optional chaining:

console.log(JSON.stringify(includeRawResponse?.raw?.kwargs?.additional_kwargs, null, 2));

If the issue persists, it might be due to how the withStructuredOutput method is implemented or called. Since the direct cause is not visible in the snippet, consider the following steps:

Review the withStructuredOutput Method: Ensure that the method correctly handles the options object and that it properly integrates with the underlying API calls. There might be a mismatch in how options are passed or handled within the library.
Check for Library Updates: There might be a bug in the version of @langchain/mistralai or @mistralai/mistralai you are using. Check the repositories or package management listings for any updates or bug fixes related to this issue.
Debugging and Logging: Add more logging around the area where additional_kwargs is expected to be accessed. This can help identify if the issue is with the response format or with how the response is being processed.

If the issue is not resolved by the above steps, consult the official documentation or reach out to the support channels for @langchain/mistralai and @mistralai/mistralai. There might be known issues or additional configuration steps required for your specific use case.

Remember, the key is to ensure that you're safely accessing properties on objects that might not always be defined, especially when dealing with asynchronous operations or external API responses.

For more context, the additional_kwargs property in the @langchain/mistralai wrapper is used to provide extra context or metadata to the language model (LLM) about the specific tool or function being invoked. In the provided context, it is used within the includeRaw option to include additional details about the tool being called, specifically when using structured output with a schema.

In the example, the additional_kwargs contains a tool_calls array, which includes details about the tool being invoked. Each item in the tool_calls array has an id, type, and a function object. The function object includes the name of the tool (in this case, "calculator") and the arguments passed to it, serialized as a JSON string. This structure allows the model to understand that it is performing a calculation and provides it with the necessary details (operation type and numbers involved) to execute the calculation and return the structured output.

This mechanism is particularly useful for applications that require the model to perform specific tasks or operations based on structured input, allowing developers to leverage the model's capabilities in a more controlled and semantically rich manner.

You can find more details about this in the examples/src/models/chat/chat_mistralai_wsa.ts file in the LangChain JS codebase.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

langchain-ai / langchainjs