Additional kwargs key prompt_token_count already exists in left dict and value has unsupported type <class 'int'> in langchain-core/utils/_merge.py merge_dict() function when running with Google gemini-1.5-pro

[RahMili]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain.js documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain.js rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

def merge_dicts(left: Dict[str, Any], right: Dict[str, Any]) -> Dict[str, Any]: """Merge two dicts, handling specific scenarios where a key exists in both dictionaries but has a value of None in 'left'. In such cases, the method uses the value from 'right' for that key in the merged dictionary.

Example:
    If left = {"function_call": {"arguments": None}} and
    right = {"function_call": {"arguments": "{\n"}}
    then, after merging, for the key "function_call",
    the value from 'right' is used,
    resulting in merged = {"function_call": {"arguments": "{\n"}}.
"""
merged = left.copy()
for right_k, right_v in right.items():
    if right_k not in merged:
        merged[right_k] = right_v
    elif right_v is not None and merged[right_k] is None:
        merged[right_k] = right_v
    elif right_v is None:
        continue
    elif type(merged[right_k]) != type(right_v):
        raise TypeError(
            f'additional_kwargs["{right_k}"] already exists in this message,'
            " but with a different type."
        )
    elif isinstance(merged[right_k], str):
        merged[right_k] += right_v
    elif isinstance(merged[right_k], dict):
        merged[right_k] = merge_dicts(merged[right_k], right_v)
    elif isinstance(merged[right_k], list):
        merged[right_k] = merge_lists(merged[right_k], right_v)
   # added this for integer
    elif isinstance(merged[right_k], int):
        merged[right_k] += right_v
    elif merged[right_k] == right_v:
        continue
    else:
        raise TypeError(
            f"Additional kwargs key {right_k} already exists in left dict and "
            f"value has unsupported type {type(merged[right_k])}."
        )
return merged

Error Message and Stack Trace (if applicable)

Error: Additional kwargs key prompt_token_count already exists in left dict and value has unsupported type <class 'int'>.

Description

Im trying to run my agents on langchain using gemini 1.5 pro as the LLM, but it is running into the above error, because in the merge_dict() function it is not expecting an integer instance and for prompt_token_count gemini returns it in integer form, also this happens only if left and right dict have different values of prompt_token_count. Manually I have added the fix and it works well, but requesting to kindly include it for us folks who are using gemini-1.5-pro, with other LLMs (non google LLMs) have not faced the same issue. Only with gemini.

System Info

windows and linux langchain-core==0.2.11 python

[dosubot[bot]]

Hey @RahMili! 👋 I'm here to help you with bug fixes, answer questions, and guide you to become a contributor. Let's tackle this issue together!

To handle integer instances correctly when merging dictionaries, specifically for the key prompt_token_count, you can modify the merge_dicts() function in langchain-core/utils/_merge.py as follows:

def merge_dicts(left: Dict[str, Any], right: Dict[str, Any]) -> Dict[str, Any]:
    """Merge two dicts, handling specific scenarios where a key exists in both
    dictionaries but has a value of None in 'left'. In such cases, the method uses the
    value from 'right' for that key in the merged dictionary.

    Example:
        If left = {"function_call": {"arguments": None}} and
        right = {"function_call": {"arguments": "{\n"}}
        then, after merging, for the key "function_call",
        the value from 'right' is used,
        resulting in merged = {"function_call": {"arguments": "{\n"}}.
    """
    merged = left.copy()
    for right_k, right_v in right.items():
        if right_k not in merged:
            merged[right_k] = right_v
        elif right_v is not None and merged[right_k] is None:
            merged[right_k] = right_v
        elif right_v is None:
            continue
        elif type(merged[right_k]) != type(right_v):
            raise TypeError(
                f'additional_kwargs["{right_k}"] already exists in this message,'
                " but with a different type."
            )
        elif isinstance(merged[right_k], str):
            merged[right_k] += right_v
        elif isinstance(merged[right_k], dict):
            merged[right_k] = merge_dicts(merged[right_k], right_v)
        elif isinstance(merged[right_k], list):
            merged[right_k] = merge_lists(merged[right_k], right_v)
        elif isinstance(merged[right_k], int) and right_k == 'prompt_token_count':
            merged[right_k] += right_v
        elif merged[right_k] == right_v:
            continue
        else:
            raise TypeError(
                f"Additional kwargs key {right_k} already exists in left dict and "
                f"value has unsupported type {type(merged[right_k])}."
            )
    return merged

This modification ensures that when merging dictionaries, if the key is prompt_token_count and both values are integers, they will be summed ^[1].

To continue talking to Dosu, mention @dosu.

[joshy-deshaw]

Upgrading to google-cloud-aiplatform==1.58.0 fixes this

[RahMili]

Thanks upgrading worked!!

[RahMili]

Closing.