dosubot[bot]
commented
1 month ago Hey @safi7! I'm here to help you with any issues you encounter with LangChain.js. I can assist you with bug fixes, answer your questions, or guide you in becoming a contributor. Let me know how I can assist you!
To handle the "max token exceed" error for anthropic models like sonnet and opus in LangChain.js, you need to implement specific error handling for these models. Here's how you can modify the completionWithRetry method to handle this error:
protected async completionWithRetry(
request: AnthropicMessageCreateParams & Kwargs,
options: AnthropicRequestOptions
): Promise<Anthropic.Message> {
if (!this.apiKey) {
throw new Error("Missing Anthropic API key.");
}
if (!this.batchClient) {
const options = this.apiUrl ? { baseURL: this.apiUrl } : undefined;
this.batchClient = new Anthropic({
...this.clientOptions,
...options,
apiKey: this.apiKey,
maxRetries: 0,
});
}
const makeCompletionRequest = async () => {
try {
return await this.batchClient.messages.create(
{
...request,
...this.invocationKwargs,
} as AnthropicMessageCreateParams,
options
);
} catch (error) {
if (error.response && error.response.data && error.response.data.error) {
const errorMessage = error.response.data.error.message;
if (errorMessage.includes("max tokens")) {
throw new Error("Max token limit exceeded for Anthropic model.");
}
}
throw error;
}
};
return this.caller.callWithOptions(
{ signal: options.signal ?? undefined },
makeCompletionRequest
);
}In this code, the makeCompletionRequest function is wrapped in a try-catch block. If the error response from the Anthropic API contains a message indicating that the maximum token limit has been exceeded, a specific error is thrown. Otherwise, the original error is re-thrown. This way, you can handle the "max token exceed" error specifically for anthropic models [1][2].
To continue talking to Dosu, mention @dosu.
jacoblee93
Checked other resources
Example Code
Error response for large content:
The system encountered an unexpected error during processing. Try your request again. at BedrockChat._streamResponseChunks (D:\development\xxx\dddd-api\node_modules\@langchain\community\dist\chat_models\bedrock\web.cjs:530:31) at processTicksAndRejections (node:internal/process/task_queues:95:5)
Error Message and Stack Trace (if applicable)
The system encountered an unexpected error during processing. Try your request again. at BedrockChat._streamResponseChunks (D:\development\xxx\dddd-api\node_modules\@langchain\community\dist\chat_models\bedrock\web.cjs:530:31) at processTicksAndRejections (node:internal/process/task_queues:95:5)
Description
Hello,
I have been using langchain js since a while. I use BedrockChat class to interact with bedrock models such as llama, mistral, sonnet 3.5 and opus. Recently, I implemented error handling for max token exceeded however, it is only working for llama and mistral now, anthropic models such as sonnet and opus returns general error. I would like to know what is the reason and how to fix. I did update the npm package and stuff. I guess there might be updates from bedrock or anthropic.
System Info
using latest @langchain/community in latest nodejs project