ChatLLM's are caches entire message object as string instead of storing the content

[Kirushikesh]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_openai import ChatOpenAI
from langchain.globals import set_llm_cache
from langchain_community.cache import SQLiteCache
from langchain_core.messages import HumanMessage

model = ChatOpenAI(model="gpt-4o-mini")
set_llm_cache(SQLiteCache(database_path=".langchain.db"))

model.invoke([HumanMessage(content="Hi! I'm Bob")])
# Cached content is '[{"lc": 1, "type": "constructor", "id": ["langchain", "schema", "messages", "HumanMessage"], "kwargs": {"content": "Hi! I\'m Bob", "type": "human"}}]'

Error Message and Stack Trace (if applicable)

No response

Description

While using the Langchain library for caching, I've observed two issues: a) The documentation primarily demonstrates examples with text completion-based LLMs for the most common caching methods (InMemory and SQLite), rather than ChatLLMs, which Langchain now recommends to use in general.

b) When using ChatLLMs with caching, the cached content is stored as a string representation of the message object, instead of the actual content within the messages. This appears to be due to the dumps() function, which converts messages into a JSON string.

To improve this, it would be beneficial if users could provide a pre-processing function, similar to the approach used in the GPTCache library. This function would convert a user message into a string. By default, for ChatLLMs, it could extract the content of the last user message. Users could also implement custom processing functions, making it easier to debug or insert specific information into the cache. I'd appreciate your thoughts on this suggestion.

System Info

"pip freeze | grep langchain"

langchain==0.2.0
langchain-anthropic==0.1.15
langchain-community==0.0.25
langchain-core==0.1.52
langchain-ibm==0.1.7
langchain-openai==0.0.5
langchain-text-splitters==0.2.1
langchainhub==0.1.14

Platform: Linux Python version: 3.10

[efriis]

Discussion on #25926 - generally I'm against this behavior, and if it's desired, caching can be implemented manually

however creates more errors than usefulness it provides imo

[Kirushikesh]

Ok got it.