search

langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
94.75k stars 15.33k forks source link

The tool is not working as expected, the observation is correct but it does not return the information the way #12467

Closed luijait closed 9 months ago

luijait commented 1 year ago

Issue you'd like to raise.

I want it to use the observation in the way I want it to use it, however it makes a summary of the observation when it is not the expected result even indicating it in the docstrings, prompt templates....

Agent: agent_chain = initialize_agent( agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, tools=tools, llm=llm, verbose=True, max_iterations=3, memory=memory )

Tool import: Tool( name='hackernews', func= get_news.run, description=""" Returns: 3 newsURL For recently news about cibersecurity if topic is not a term of the follow list, put the most accurate term along this list like a topic Args: topic -> the only topics valids are general, dataBreach, cyberAttack, vulnerability, malware, security, cloud, tech, iot, bigData, business, mobility, research, corporate, socialMedia. Useful for find rencently news about computer science, cybersecurity... Returns: Links to the related news and a description of them""" ), """

Code of tool: """ from cybernews.cybernews import CyberNews from langchain.tools import tool news = CyberNews() import subprocess import requests import dns.resolver from pydantic import BaseModel, Extra

class get_news(BaseModel): """Herramientas de ciberseguridad Returns: news, you shoul output newsURL and the description of that url """

class Config:
    """Configuration for this pydantic object."""
    extra = Extra.forbid

def run(self, topic: str) -> str:
    """
Busca noticias de ciberseguridad

Args:
topic (str): noticia a encontrar, tema a encontrar

Returns:
news.get_news(topic) (list): a list who contain description and links, you should provide a few links about the lastest news
"""
    if topic == "":
        topic = "cybersecurity"
        return(news.get_news(topic))
    else:
        return(news.get_news(topic))

"""" Output: Screenshot 2023-10-28 at 01 57 46

this just provide a short description but no links, how I can do it, all my tools have the same problem... don't output like is defined in prompts....

Suggestion:

No response

dosubot[bot] commented 1 year ago

🤖

Hello @luijait,

I understand that you're having an issue with the LangChain framework where it's summarizing the observations instead of using them as defined in the docstrings and prompt templates. I'm here to help you with this.

Based on the context you've provided, it seems like the issue might be related to the PromptTemplate object in the LangChain framework. This object is used to generate prompts for the language model, and it seems like it's currently set to summarize the text.

If you want the language model to use the text as it is, you can modify the PromptTemplate object. Here's an example of how you can do this:

prompt_template = """Use the following as it is:

"{text}"

DO NOT SUMMARIZE:"""
PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])

In this modified prompt_template, the instruction for the language model is changed from summarizing the text to using the text as it is. This might help in getting the expected output from the language model.

You can find more information about the PromptTemplate object in the LangChain framework in the following files:

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

luijait commented 1 year ago

🤖

Hello @luijait,

I understand that you're having an issue with the LangChain framework where it's summarizing the observations instead of using them as defined in the docstrings and prompt templates. I'm here to help you with this.

Based on the context you've provided, it seems like the issue might be related to the PromptTemplate object in the LangChain framework. This object is used to generate prompts for the language model, and it seems like it's currently set to summarize the text.

If you want the language model to use the text as it is, you can modify the PromptTemplate object. Here's an example of how you can do this:

prompt_template = """Use the following as it is:

"{text}"

DO NOT SUMMARIZE:"""
PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])

In this modified prompt_template, the instruction for the language model is changed from summarizing the text to using the text as it is. This might help in getting the expected output from the language model.

You can find more information about the PromptTemplate object in the LangChain framework in the following files:

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

It won't work, agent_chain.agent.llm_chain.prompt.messages[0].prompt.template = """ Prompt

DO NOT SUMMARIZE:""" still this output Screenshot 2023-10-28 at 02 14 58

dosubot[bot] commented 1 year ago

🤖

Based on the context provided, it seems like the issue you're facing might be due to the SENTENCE_PROMPT template not being used in your agent chain. The SENTENCE_PROMPT template is designed to create a detailed and interesting sentence using all the given fields and preferences. If the agent chain is using the SUMMARY_PROMPT or ENTITY_SUMMARIZATION_PROMPT, it will summarize the observations.

To resolve this issue, you need to ensure that the SENTENCE_PROMPT template is being used in the agent chain. Here's how you can do this:

  1. Check the agent chain: Make sure that the agent chain is using the SENTENCE_PROMPT template. If it's using the SUMMARY_PROMPT or ENTITY_SUMMARIZATION_PROMPT, it will summarize the observations. If you want to use the observations as defined, you should use the SENTENCE_PROMPT.

  2. Check the PromptTemplate modifications: If you have modified the PromptTemplate object, make sure that these modifications are correct and are being applied. If the modifications are incorrect or not being applied, the framework might still be using the default templates, which could be causing the issue.

  3. Check the input variables: The input variables for the SENTENCE_PROMPT template should match the variables that are being passed in. If there's a mismatch, the template might not be used correctly, which could lead to the observations being summarized.

  4. Debug the code: If you're still facing the issue, you might need to debug the code to identify where the issue is occurring. This could involve adding print statements to check the values of variables, using a debugger to step through the code, or checking the logs if any are available.

In the context shared, it seems like you're using a custom PromptTemplate with the variable "question". Make sure that this variable is being passed correctly and that the template is being used in the agent chain. If you're using a chat model, the CHAT_PROMPT will be used instead, which also uses the "question" variable.

For more details, you can refer to the following sources:

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

luijait commented 1 year ago

🤖

Based on the context provided, it seems like the issue you're facing might be due to the SENTENCE_PROMPT template not being used in your agent chain. The SENTENCE_PROMPT template is designed to create a detailed and interesting sentence using all the given fields and preferences. If the agent chain is using the SUMMARY_PROMPT or ENTITY_SUMMARIZATION_PROMPT, it will summarize the observations.

To resolve this issue, you need to ensure that the SENTENCE_PROMPT template is being used in the agent chain. Here's how you can do this:

  1. Check the agent chain: Make sure that the agent chain is using the SENTENCE_PROMPT template. If it's using the SUMMARY_PROMPT or ENTITY_SUMMARIZATION_PROMPT, it will summarize the observations. If you want to use the observations as defined, you should use the SENTENCE_PROMPT.
  2. Check the PromptTemplate modifications: If you have modified the PromptTemplate object, make sure that these modifications are correct and are being applied. If the modifications are incorrect or not being applied, the framework might still be using the default templates, which could be causing the issue.
  3. Check the input variables: The input variables for the SENTENCE_PROMPT template should match the variables that are being passed in. If there's a mismatch, the template might not be used correctly, which could lead to the observations being summarized.
  4. Debug the code: If you're still facing the issue, you might need to debug the code to identify where the issue is occurring. This could involve adding print statements to check the values of variables, using a debugger to step through the code, or checking the logs if any are available.

In the context shared, it seems like you're using a custom PromptTemplate with the variable "question". Make sure that this variable is being passed correctly and that the template is being used in the agent chain. If you're using a chat model, the CHAT_PROMPT will be used instead, which also uses the "question" variable.

For more details, you can refer to the following sources:

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

how can I see what prompt im using?

dosubot[bot] commented 1 year ago

🤖

Based on the context provided, it seems like you're trying to determine which prompt template is being used in your agent chain. In the LangChain framework, the prompt template being used in your agent chain is determined by the PromptTemplate class from the langchain.prompts module. This class takes a list of input variables and a template string to create a prompt template.

In your case, there is one prompt template defined: PROMPT. The PROMPT template is used for creating a friendly conversation between a human and an AI, where the AI provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. Therefore, the prompt template being used in this agent chain is the PROMPT template.

To check the formatted prompt being used, you can use the chain.run() method and pass the variables as a dictionary. Here is an example:

from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI
import os

os.environ["OPENAI_API_KEY"] = "sk- Your-API-KEY”

llm = OpenAI(temperature=0.9)

prompt = PromptTemplate(
  input_variables=["Brand", "Product"],
  template="What would be the name of {Brand} that sells {Product}?",
)

from langchain.chains import LLMChain

chain = LLMChain(llm=llm, prompt=prompt)

print(chain.run({
  'Brand': "Art supplies",
  'Product': "colors"
}))

In this example, the chain.run() method is used to get the formatted prompt. You just need to pass the variables as a dictionary to the run() method.

As for the "DO NOT SUMMARIZE" directive in the PromptTemplate object, I wasn't able to find specific information on how the LangChain framework handles this directive. It's possible that the framework doesn't recognize this directive, or it may be handled in a part of the code that I wasn't able to find. If you could provide more details about where you're seeing this issue, I might be able to provide a more specific answer.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

dosubot[bot] commented 1 year ago

@baskaryan, I need your expertise on this issue.

luijait commented 1 year ago

@baskaryan, I need your expertise on this issue.

y can someone help me pls?

dosubot[bot] commented 9 months ago

Hi, @luijait,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you raised an issue regarding a tool not returning the expected information despite the correct observation. The issue involves the LangChain framework and the use of the PromptTemplate object. There have been responses from me providing detailed explanations and code examples on modifying the PromptTemplate object and ensuring the correct template is being used in the agent chain. You also requested assistance from @baskaryan.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you!