search

langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
92.4k stars 14.77k forks source link

HumanMessage error expecting a str type in content #11882

Closed pereaf13 closed 7 months ago

pereaf13 commented 11 months ago

System Info

Python 3.11.6 Langchain 0.0.315

Device name Precision7760 Processor 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz 2.61 GHz Installed RAM 32.0 GB (31.2 GB usable) Device ID 049EB0D9-D534-47A1-9F59-62B1F3D578D4 Product ID 00355-60713-95419-AAOEM System type 64-bit operating system, x64-based processor Pen and touch No pen or touch input is available for this display

Edition Windows 11 Pro Version 22H2 Installed on ‎10/‎7/‎2023 OS build 22621.2428 Experience Windows Feature Experience Pack 1000.22674.1000.0

Who can help?

@hwchase17 @agola11

Information

Related Components

Reproduction

from langchain.schema import AIMessage, HumanMessage from langchain.chat_models import ChatOpenAI from langchain.memory import ConversationBufferMemory from langchain.chains import ConversationChain

chat = ChatOpenAI() memory = ConversationBufferMemory() conversation = ConversationChain(llm=chat, memory=memory)

user_input="Hola atenea yo me llamo franks y estoy interesado en adquirir un vehículo"

response = conversation.run([HumanMessage(content=str(user_input))])

print(response)

I am getting this errors: Traceback (most recent call last): File "F:\Audi\Chatbot_Demo\mini_example.py", line 12, in response = conversation.run([HumanMessage(content=str(user_input))]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Python\Python311\Lib\site-packages\langchain\chains\base.py", line 503, in run return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Python\Python311\Lib\site-packages\langchain\chains\base.py", line 310, in call final_outputs: Dict[str, Any] = self.prep_outputs( ^^^^^^^^^^^^^^^^^^ File "E:\Python\Python311\Lib\site-packages\langchain\chains\base.py", line 406, in prep_outputs self.memory.save_context(inputs, outputs) File "E:\Python\Python311\Lib\site-packages\langchain\memory\chat_memory.py", line 36, in save_context self.chat_memory.add_user_message(input_str) File "E:\Python\Python311\Lib\site-packages\langchain\schema\chat_history.py", line 46, in add_user_message self.add_message(HumanMessage(content=message)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Python\Python311\Lib\site-packages\langchain\load\serializable.py", line 97, in init super().init(kwargs) File "E:\Python\Python311\Lib\site-packages\pydantic\v1\main.py", line 341, in init raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for HumanMessage content str type expected (type=type_error.str)**

Expected behavior

The argument passed to HumanMessage is a str, nevertheless an error is produced (as it wasn't).

dosubot[bot] commented 11 months ago

🤖

Hello,

Thank you for providing detailed information about the issue you're facing. From the traceback you've provided, it seems that the error is occurring when the HumanMessage instance is being created. The HumanMessage class expects the content attribute to be a string, as defined in the BaseMessage class from which HumanMessage inherits:

class BaseMessage(Serializable):
    """The base abstract Message class.
    Messages are the inputs and outputs of ChatModels.
    """
    content: str
    """The string contents of the message."""
    ...

In your code, you're passing a string to HumanMessage:

response = conversation.run([HumanMessage(content=str(user_input))])

This should work as expected, as user_input is a string and you're explicitly converting it to a string again with str(user_input).

However, the error message suggests that HumanMessage is not receiving a string. This could be due to some unexpected behavior in the ConversationChain.run() method or in the HumanMessage class itself.

I found a similar issue in the LangChain repository: add_user_message() fails. called with List instead of a string.. This issue was resolved, and it might be related to the problem you're facing.

To help you further, could you please provide the following additional information:

  1. The exact version of LangChain you're using. You mentioned that it's 0.0.315, but please confirm if this is the exact version.
  2. Any modifications you've made to the official LangChain code, if any.
  3. The full code of your script, if possible. This will help me understand the context in which HumanMessage is being used.

Once I have this information, I'll be able to assist you better. Thank you for your patience.

Best regards.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

pereaf13 commented 11 months ago

0.0.315 is correct, it is what is installed in my system, I have not mande any modification.

The full code is as below. The objective is to set up a chatbot that is able to interact with the user by spoken language. A previous version without Langchain was working ok (not finished, though), but I decided to use Langchain in order to take advantage of things like automated handling of the conversation history size... I strongly believe the problem is not in my code, but rather in the library. I took the courses "LangChain for LLM Application Development" and "LangChain: Chat with Your Data" so I tried my small example in the notebook. I got the same result. So I don't think its either my system.

Code:

import openai import speech_recognition as sr import pyttsx3 import logging import tkinter as tk from tkinter import scrolledtext import threading import json import re

from langchain.schema import AIMessage, HumanMessage from langchain.chat_models import ChatOpenAI from langchain.memory import ConversationBufferMemory from langchain.chains import ConversationChain

Global variables

source = sr.Microphone() rec = sr.Recognizer()

openai.api_key = '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' # Replace with your API key

Load the cleaned JSON data into a dictionary

with open("models_versions_tech_specs_equipment.json", "r", encoding='utf-8') as file: models_versions_tech_specs_equipment = json.load(file)

Set up logging

logging.basicConfig(level=logging.INFO, filename="chatbot.log", filemode="w", format="%(asctime)s - %(levelname)s - %(message)s", encoding='utf-8')

Initialize text-to-speech engine

engine = pyttsx3.init() voices = engine.getProperty('voices') female_voice_id = voices[3].id # Sabina de Mexico engine.setProperty('voice', female_voice_id)

Initialize langchain components

chat = ChatOpenAI() memory = ConversationBufferMemory() conversation = ConversationChain(llm=chat, memory=memory)

Initial context for the chatbot

-------------------------------------------------------------------------------------------------------------

Updated System Role

system_role = { 'role': 'system', 'content': ('You are "Atenea", an assistant designed to help users with the purchase process of new Audi vehicles in Mexico. ' 'In this environment, you interact with two main entities: the User and the Database. ' 'You have inherent knowledge, but for accurate and up-to-date Audi vehicle details, you should query the Database using fetch_commands. ' 'Responses from the Database will be clearly indicated with a prefix "[DB Response]". Always prioritize data from the Database over your inherent knowledge. ' 'Your primary goal is to provide accurate information, understand the user’s needs, and guide them towards one of the outcomes: a test drive, in-person meeting, or a follow-up. ' 'Balance between being proactive in needs detection and addressing the user’s direct queries. Prioritize the user’s intent and always offer a patient, courteous interaction.') }

Updated Context

context = { "vehicle_db_interaction": { "purpose": "Guide the chatbot in interacting with the vehicle database hierarchically.",

    "data_hierarchy": {
        "level_1": "Model Names",
        "level_2": "Model Years for a specific model",
        "level_3": "Versions for a specific model and year",
        "level_4": "Categories of features for a specific model, year, and version",
        "level_5": "Features and their descriptions for a specific model, year, version, and category"
    },

    "hierarchical_access": {
        "levels": {
            "level_1": {
                "name": "Model Names",
                "command": "fetch_model_list",
                "description": "To fetch the list of available models, issue the highest level command without any parameters.",
                "required_parameters": []
            },
            "level_2": {
                "name": "Model Years for a specific model",
                "command": "fetch_years_for_model",
                "description": "To get the years available for a specific model, you need to have requested the first level (models). Issue the second level command with the model name as a parameter.",
                "required_parameters": ["model"]
            },
            "level_3": {
                "name": "Versions for a specific model and year",
                "command": "fetch_versions_for_model_and_year",
                "description": "To fetch versions for a specific model and year, first get model names and years. Then, issue the third level command with 'model' and 'year' as parameters.",
                "required_parameters": ["model", "year"]
            },
            "level_4": {
                "name": "Categories of features for a specific model, year, and version",
                "command": "fetch_categories_for_version",
                "description": "To retrieve feature categories, ensure you have model, year, and version data. Use the model, year, and version as parameters.",
                "required_parameters": ["model", "year", "version"]
            },
            "level_5": {
                "name": "Features and their descriptions for a specific model, year, version, and category",
                "command": "fetch_features_for_category",
                "description": "For detailed feature access, first get data for model, year, version, and category. Then, issue this command with the model, year, version, and category as parameters.",
                "required_parameters": ["model", "year", "version", "category"]
            }
        }
    },

    "fetch_commands": {
        "fetch_model_list": {
            "description": "Retrieves the list of all available Audi models in the Mexican market.",
            "usage": '{"command": "fetch", "details": {"action": "available_models"}}',
        },
        "fetch_years_for_model": {
            "description": "Retrieves the years available for a specific Audi model.",
            "usage": '{"command": "fetch", "details": {"action": "years_for_model", "model": "[MODEL_NAME]"}}'
        },
        "fetch_versions_for_model_and_year": {
            "description": "Retrieves the versions available for a specific Audi model for a given year.",
            "usage": '{"command": "fetch", "details": {"action": "versions_for_model_and_year", "model": "[MODEL_NAME]", "year": "[YEAR]"}}'
        },
        "fetch_categories_for_version": {
            "description": "Retrieves the categories of features for a specific version of an Audi model for a given year.",
            "usage": '{"command": "fetch", "details": {"action": "categories_for_version", "model": "[MODEL_NAME]", "year": "[YEAR]", "version": "[VERSION_NAME]"}}'
        },
        "fetch_features_for_category": {
            "description": "Retrieves the features for a specific category of a specific version of an Audi model for a given year.",
            "usage": '{"command": "fetch", "details": {"action": "features_for_category", "model": "[MODEL_NAME]", "year": "[YEAR]", "version": "[VERSION_NAME]", "category": "[CATEGORY]"}}'
        },
        "reminder": ("Only refer to the models provided in the 'available_models' list. "
                     "Do not mention or provide details on models not present in this list. "
                     "Before offering any details or specifications about an Audi vehicle, always verify the information against the 'available_models' context.")
    },

    "vehicle_type": {
        "electric": "Models with the name 'e-tron' or 'etron' are electric.",
        "ice": "The default assumption is that models are Internal Combustion Engines (ICE) unless specified otherwise.",
        "hybrid": "Currently, there are no hybrid models available."
    },

    "model_descriptions": {
        "SUV": "Models starting with a Q, like Q5, are SUVs.",
        "Sedan": "Models starting with an A, like A4, are sedans.",
        "Sport": "Models starting with an S, like S5, are sportive versions of the base models.",
        "High-Performance": "Models starting with RS, like RS5, are the next level of sportiness, above S models."
    },

    "equipment_status": {
        "default": "Serie means the equipment is always included in this version.",
        "optional": "Opcional means the equipment can be configured at the time of ordering and will have an extra cost.",
        "not_available": "No disponible means the equipment is neither included by default nor can be configured."
    },

    "user_intent_guidelines": {
        "information": "Recognize when the user is seeking details about a specific model or feature. Respond with accurate and detailed information.",
        "comparison": "Understand when a user is trying to compare models or features. Provide clear and concise comparisons.",
        "purchase": "Detect when a user is inclined towards making a purchase. Guide them towards the desired outcomes."
    },

    "enhanced_needs_principles": {
        "driving_environment": "Determine if the user prefers city or highway driving, and factor this into model suggestions.",
        "lifestyle": "Understand the user's daily activities and needs. Is the car for work, recreation, or specific activities?",
        "performance": "Gauge if the user prioritizes power, fuel efficiency, or a balance of both."
    },

    "data_integrity_guidelines": {
        "always_validate": "Always check data sources before providing information. Never assume.",
        "data_source": "Use the data provided through the fetch_commands as the primary source of information."
    },

    "user_friendly_principles": {
        "acknowledge": "Always acknowledge user queries before providing a response.",
        "clarify": "If unsure, seek clarity rather than making assumptions.",
        "converse": "Engage in a conversational manner, ensuring the user feels heard and assisted."
    },

    "interaction_guidelines": {
        "initial_interaction": [
            "¡Hola! Es un placer atenderte.",
            "¿Qué te trae por aquí hoy?"
        ],
        "current_vehicle": [
            "¿Qué coche manejas actualmente?",
            "¿Por qué estás considerando un cambio?"
        ],
        "usage_and_preferences": [
            "¿Quién conducirá el vehículo principalmente?",
            "¿Viajarás solo o con pasajeros?",
            "¿Cómo usarás el vehículo?",
            "¿Conduces más en ciudad o en carretera?",
            "¿Qué importancia le das a factores como el consumo de combustible, la potencia, y la seguridad?",
            "¿Tienes alguna preferencia sobre los interiores y la transmisión?"
        ],
        "financial_capacity": [
            "¿Cuál es tu presupuesto mensual o rango de presupuesto que tienes en mente?"
        ],
        "vehicle_body_preference": [
            "¿Tienes alguna preferencia por el tipo de carrocería del vehículo? Por ejemplo, ¿prefieres un sedan o un SUV?"
        ],
        "emotional_factors": [
            "Cuéntame, ¿hay algún momento especial o actividad que te gustaría experimentar con tu nuevo Audi?",
            "¿Qué es lo primero que te imaginas haciendo con tu nuevo coche?",
            "Para ti, ¿qué hace que un coche sea más que solo un medio de transporte?"
        ],
        "post_interaction": [
            "¡Gracias por tu interés en Audi! Si tienes más dudas, aquí estaré.",
            "Si cambias de opinión o tienes más preguntas, no dudes en regresar."
        ]
    },

    "desired_outcomes": {
        "test_drive": [
            "Basado en tus respuestas, creo que te beneficiaría probar [Model Name]. ¿Te gustaría agendar una prueba de manejo?",
            "Siento que [Model Name] sería perfecto para ti. ¿Qué te parece si programamos una prueba de manejo?"
        ],
        "call_back": [
            "Entiendo que aún necesitas tiempo para pensar. ¿Qué te parece si te llamo en [XX days] para seguir conversando sobre tu elección?",
            "Sé que tomar una decisión puede tomar tiempo. ¿Te gustaría que nos pongamos en contacto contigo en unos días?"
        ],
        "human_salesman": [
            "Si prefieres discutir tus opciones en detalle con un experto en ventas, puedo programar una cita para ti. ¿Te gustaría?",
            "Entiendo que algunas decisiones requieren una conversación en profundidad. ¿Quieres que te ponga en contacto con uno de nuestros especialistas en ventas?"
        ]            
    }
}

}

The extraction of available models and adding them to the chatbot's context remains unchanged:

available_models = list(models_versions_tech_specs_equipment.keys()) context["vehicle_db_interaction"]["available_models"] = available_models

The conversation history initialization also remains unchanged.

conversation_history = [ system_role, { "role": "system", "content": "Hola. Soy Atenea, tu asistente personalizado de Ventas para vehículos Audi nuevos en México. Dime, ¿cómo te puedo ayudar hoy?" } ]

Utility Functions

-------------------------------------------------------------------------------------------------------------

def preprocess_response(response_text): """Clean up the response to make it suitable for speech using regex for numbered lists."""

# Replace double asterisks
response_text = response_text.replace("**", "")

# Replace numbered lists with conversational starters
def replace_numbered_list(match):
    num = int(match.group(1))
    starters = ["Primero", "Segundo", "Tercero", "Cuarto", "Quinto", 
                "Sexto", "Séptimo", "Octavo", "Noveno", "Décimo"]
    # For numbers beyond the list, just use the number
    return starters[num-1] if num <= len(starters) else str(num)

response_text = re.sub(r'(\d+)\.', replace_numbered_list, response_text)

# Additional replacements for conversational speech
response_text = response_text.replace("(", "")
response_text = response_text.replace(")", "")
response_text = response_text.replace("[", "")
response_text = response_text.replace("]", "")
response_text = response_text.replace(":", ",")
response_text = response_text.replace(";", ",")
response_text = response_text.replace("-", " a ")
response_text = response_text.replace("&", "y")
response_text = response_text.replace(" e.g. ", " por ejemplo ")
response_text = response_text.replace(" i.e. ", " es decir ")

return response_text

Data Retrieval Functions

-------------------------------------------------------------------------------------------------------------

def fetch_model_list(): """Function to return a list of available models.""" model_list = list(models_versions_tech_specs_equipment.keys()) logging.info(f"fetch_model_list() called. Result: {model_list}") models_str = ', '.join(model_list) return f"[DB Response]: The available Audi models in the Mexican market are: {models_str}."

def fetch_years_for_model(model_name): """Retrieve the available years for a specific model.""" if model_name in models_versions_tech_specs_equipment: years = list(models_versions_tech_specs_equipment[model_name].keys()) logging.info(f"fetch_years_for_model({model_name}) called. Result: {years}") years_str = ', '.join(years) return f"[DB Response]: For the {model_name} model, the available years are: {years_str}." return f"[DB Response]: No years found for the {model_name} model."

def fetch_versions_for_model_and_year(model_name, year): """Function to return available versions for a given model and year.""" if model_name in models_versions_tech_specs_equipment and year in models_versions_tech_specs_equipment[model_name]: versions = list(models_versions_tech_specs_equipment[model_name][year].keys()) logging.info(f"fetch_versions_for_model_and_year({model_name}, {year}) called. Result: {versions}") versions_str = ', '.join(versions) return f"[DB Response]: For the {model_name}, Model Year {year}, the available versions are: {versions_str}." return f"[DB Response]: No versions found for the {model_name} model, year {year}."

def fetch_categories_for_version(model_name, year, version_name): """Retrieve the available categories for a specific model, year, and version.""" if (model_name in models_versions_tech_specs_equipment and year in models_versions_tech_specs_equipment[model_name] and version_name in models_versions_tech_specs_equipment[model_name][year]): categories = list(models_versions_tech_specs_equipment[model_name][year][version_name].keys()) logging.info(f"fetch_categories_for_version({model_name}, {year}, {version_name}) called. Result: {categories}") categories_str = ', '.join(categories) return f"[DB Response]: For the {model_name}, Model Year {year}, Version {version_name}, the feature categories are: {categories_str}." return f"[DB Response]: No feature categories found for the {model_name} model, year {year}, version {version_name}."

def fetch_features_for_category(model_name, year, version_name, category): """Retrieve the features and their values for a specific model, year, version, and category.""" if (model_name in models_versions_tech_specs_equipment and year in models_versions_tech_specs_equipment[model_name] and version_name in models_versions_tech_specs_equipment[model_name][year] and category in models_versions_tech_specs_equipment[model_name][year][version_name]): features = models_versions_tech_specs_equipment[model_name][year][version_name][category] logging.info(f"fetch_features_for_category({model_name}, {year}, {version_name}, {category}) called. Result: {features}") features_str = ', '.join([f"{key}: {value}" for key, value in features.items()]) return f"[DB Response]: For the {model_name}, Model Year {year}, Version {version_name}, Category {category}, the features are: {features_str}." return f"[DB Response]: No features found for the {model_name} model, year {year}, version {version_name}, category {category}."

Main Interaction Functions

-------------------------------------------------------------------------------------------------------------

def listen_to_user(): global chatbox, source, rec

try:
    with source:  # Use the global source variable
        chatbox.insert(tk.END, "\nEscuchando...\n")
        chatbox.see(tk.END)
        audio = rec.listen(source)
    user_message = str(rec.recognize_google(audio, language='es-MX'))
    chatbox.insert(tk.END, f"\nUsuario: {user_message}\n", 'user')
    chatbox.see(tk.END)
    return user_message
except sr.UnknownValueError:
    logging.error("Google Web Speech API could not understand audio")
    return "Lo siento, no pude reconocer tu mensaje."
except sr.RequestError as e:
    logging.error(f"Could not request results from Google Web Speech API; {e}")
    return "Lo siento, hubo un problema al conectar con el servicio de reconocimiento de voz."

def speak_to_user(text): global engine chatbox.insert(tk.END, f"\nAtenea: {text}\n", 'bot') chatbox.see(tk.END) engine.say(text) engine.runAndWait()

def detect_fetch_command(response_text): """Detect and extract fetch commands and DB responses from the response text."""

Detecting a database response

if "Database Response:" in response_text:
    return "db_response", response_text.replace("Database Response:", "").strip()

# Detecting a fetch command
try:
    parsed_response = json.loads(response_text)
    if "command" in parsed_response and parsed_response["command"] == "fetch":
        details = parsed_response.get("details", {})
        return details.get("action"), details
except json.JSONDecodeError:
    return None, {}
return None, {}

def main_interaction(user_input): global conversation

# Get the chatbot's response using the langchain library
logging.info(f"Type of user_input: {type(user_input)}, Value: {user_input}")
response = conversation.run([HumanMessage(content=str(user_input))])
chatbot_response = response[-1].content

logging.info(f"Chatbot raw response: {chatbot_response}")
action, details = detect_fetch_command(chatbot_response)

if action == "db_response":
    # Return the DB response as a message for continuity, but the chatbot should act on this data
    return details
elif action:
    model = details.get("model")
    year = details.get("year")
    version = details.get("version")
    category = details.get("category")

    # Call the respective function based on the action
    if action == "available_models":
        return fetch_model_list()
    elif action == "years_for_model":
        return fetch_years_for_model(model)
    elif action == "versions_for_model_and_year":
        return fetch_versions_for_model_and_year(model, year)
    elif action == "categories_for_version":
        return fetch_categories_for_version(model, year, version)
    elif action == "features_for_category":
        return fetch_features_for_category(model, year, version, category)
    else:
        return "Invalid fetch action."
else:
    return chatbot_response

Thread and GUI Handling Functions

-------------------------------------------------------------------------------------------------------------

def start_chat(event=None): global rec, source # Declare the source variable as global

with source:
    chatbox.insert(tk.END, "\nAjustando para el ruido ambiental. Por favor, espera...\n")
    chatbox.see(tk.END)
    rec.adjust_for_ambient_noise(source, duration=5)

speak_to_user("Hola. Soy Atenea, tu asistente personalizado de Ventas para vehículos Audi nuevos en México. Dime, ¿cómo te puedo ayudar hoy?")
threading.Thread(target=threaded_chat_interaction).start()  # Run the chat interaction in a separate thread

def threaded_chat_interaction(): """Wrapper function to run the chat interaction in a separate thread.""" while True: user_message = listen_to_user() if "salir" in user_message.lower(): speak_to_user("Hasta luego!") break

    chatbot_response = main_interaction(user_message)
    speak_to_user(chatbot_response)

GUI setup and execution

-------------------------------------------------------------------------------------------------------------

root = tk.Tk() root.title("Atenea - Asistente Audi") chatbox = scrolledtext.ScrolledText(root, wrap=tk.WORD, width=80, height=30) chatbox.pack(padx=10, pady=10) chatbox.tag_configure('user', foreground='blue') chatbox.tag_configure('bot', foreground='green') start_button = tk.Button(root, text="Iniciar Chat", command=start_chat) start_button.pack(pady=10)

root.mainloop()

dosubot[bot] commented 7 months ago

Hi, @pereaf13,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you reported an error related to the HumanMessage class in the langchain library, where it's expecting a string type in the content but the provided input is not being recognized as a string, resulting in a validation error. I have responded with an analysis of the issue, suggesting potential reasons for the error and requesting additional information to further investigate the problem. You confirmed the version of LangChain being used and provided the full code of the script. The conversation is ongoing, with me seeking to understand the context in which HumanMessage is being used to assist further.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you!

meiramz commented 3 months ago

any update on this one? having the same issue.