Getting "RuntimeError: Tool context error detected. This can occur due to parallelization." while invoking a chain using langchain_google_vertexai

dianz17 commented 2 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

retrieve_chain = self.cat_query_prompt | self.retriever_llm | StrOutputParser()

json_s = retrieve_chain.invoke({"question": query,"schema":self.cat_schema_json, "brand_list":list(df['brand_name'].unique()), "few_shot":self.cat_schema_fewshot})

Error Message and Stack Trace (if applicable)

File "/workspace/selfquery_utils.py", line 151, in get_categorical_filters json_s = retrieve_chain.invoke({"question": query,"schema":self.cat_schema_json, "brand_list":list(df['brand_name'].unique()), "few_shot":self.cat_schema_fewshot}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2499, in invoke input = step.invoke( ^^^^^^^^^^^^ File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 276, in invoke self.generate_prompt( File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 633, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 803, in generate output = self._generate_helper( ^^^^^^^^^^^^^^^^^^^^^^ File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 670, in _generate_helper raise e File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 657, in _generate_helper self._generate( File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_google_vertexai/llms.py", line 223, in _generate res = _completion_with_retry( ^^^^^^^^^^^^^^^^^^^^^^^ File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_google_vertexai/llms.py", line 72, in _completion_with_retry with telemetry.tool_context_manager(llm._user_agent): File "/layers/google.python.runtime/python/lib/python3.11/contextlib.py", line 144, in exit next(self.gen) File "/layers/google.python.pip/pip/lib/python3.11/site-packages/google/cloud/aiplatform/telemetry.py", line 48, in tool_context_manager _pop_tool_name(tool_name) File "/layers/google.python.pip/pip/lib/python3.11/site-packages/google/cloud/aiplatform/telemetry.py", line 57, in _pop_tool_name raise RuntimeError( RuntimeError: Tool context error detected. This can occur due to parallelization.

Description

The chain generates a json that is goint to be used as a filter for a vector database. I've been using this chain for months and its the first time that I get this error. I tried to replicate it but nothing happened.

System Info

google-cloud-discoveryengine==0.11.2 google-cloud-aiplatform langchain langchain-core langchain-experimental langchainplus-sdk langchain-google-genai ipywidgets==7.7.2 pandas==2.0.3 google-cloud-bigquery db-dtypes langchain-google-vertexai shortuuid google-cloud-storage redis

shenghann commented 2 months ago

I'm seeing this issue on langchain_google_vertexai as well, when try to run a few chains in a RunnableParallel. Seems to be intermittent in my case.

System info: google-cloud-discoveryengine==0.11.11 google-cloud-aiplatform==1.48.0 langchain==0.0.354 langchain-core==0.1.45 langchain-community==0.0.20 langchain-google-vertexai==0.1.2

casbra13 commented 2 months ago

I had the same problem and could fix it by upgrading the langchain-google-vertexai package. My version now is 1.0.2.

asketos commented 2 months ago

The same issue, updating to langchain-google-vertexai==0.1.2 doesn't help. Here's my simplest example demonstrating how to reproduce the same error:

from langchain_google_vertexai import VertexAI

prompt = "1 + 2 = ?"
models = [
    "gemini-1.5-pro-preview-0409",
    "gemini-1.0-pro",
    "text-bison-32k",
]
tasks = []

for model_name in models:
    llm = VertexAI(
        model_name=model_name,
        project=project,
    )
    tasks.append(llm.ainvoke(prompt))

results = await asyncio.gather(*tasks)
print(results)

jswortz commented 1 month ago

Wanted to add we are facing the same issue with the same test above (python 3.11) and the new gemini-experimental llm (same as other llms listed above)

jswortz commented 1 month ago

Quick update - I did some investigation and the error was being caused by the `_pop_tool_name in aiplatform==1.5.0's context manager. I think this has to do with the gemini tool names. There may need to be a long term fix to accommodate for Gemini tool name schemas

For now, I was able to fix this by patching the 1.5.0 file per below:

Update I was able to successfully get around this error with google-cloud-aiplatform==1.51.0 but still needed the code change to google.cloud.aiplatform.telemetry

# in google.cloud.aiplatform.telemetry.py#L55
def _pop_tool_name(tool_name: str) -> None:
    if not _tool_names_to_append or _tool_names_to_append[-1] != tool_name:
        return None
    _tool_names_to_append.pop()

jswortz commented 1 month ago

One more update - it seems the issue for me at least is the VertexAIEmbeddings library langchain_google_vertexai .

The code works fine for me by using the deprecated from langchain_community.embeddings import VertexAIEmbeddings (with a deprecation warning)

It seems that the langchain_google_vertexai needs to leverage the working code from langchain_community.embeddings

Does the example work above if you change the import of from langchain_community.llms import VertexAI ?

abetatos commented 1 month ago

I opened an issue in aiplatform as I think the problem originates there, hope you dont mind!

Freezaa9 commented 4 weeks ago

Look like it is fixed: https://github.com/googleapis/python-aiplatform/issues/3838 in google-cloud-aiplatform == 1.54.0

langchain-ai / langchain