Functionality for prompts based on decision tree / flow charts.

[jsbryaniv]

Feature request

I want to request functionality for decision tree / flow chart like prompt architecture. The idea is that there would be a Prompt Tree that starts on a specific branch then allows the LLM to select new branches as part of its toolkit. Each branch would have its own prompts meaning that the AI does not need to be given all the information up front and instead can break down its commands into bite sized chunks that it sees at each branch in the tree.

Motivation

This would help chatbot workflows by limiting the amount of information the LLM sees at each point in time, it can collect variables through different branches of the tree that it will use later, and it would improve reliability with LLM outputs because it would be easier to implement checks. It also could eliminate the need for a scratchpad, which can become costly if abused by the LLM.

Also, this is a feature that is available in other systems such as LLMFlows and Amazon Lex. And from what I have seen it is frequently on message boards here.

Your contribution

I have made a simple example script to show how this could work in principle. However, I do not have experience contributing to open source projects so I am not sure what formatting mistakes I may be making, nor where exactly in the object heirarchy this should belong (is this a type of Prompt? Or Agent?). I would love to learn about what is needed to incorporate this into the LangChain functionality.

In my example I make a PromptTree class which stores the state and can access the current prompt. Inside the tree are a variety of branches which point to eachother according to a dictionary. Each branch produces a tool which allows the AI to switch branches by updating the prompttree.

# Import libraries
import ast
from pydantic.v1 import BaseModel, Field
from langchain.tools import Tool
from langchain.schema import HumanMessage, AIMessage, SystemMessage, FunctionMessage
from langchain.tools import format_tool_to_openai_function
from langchain.chat_models import ChatOpenAI

### Define PromptBranch ###

# Declare function name variable
SELECT_BRANCH = 'select_branch'
UPDATE_INSIGHT = 'update_insight'

# Create PromptTreeBranch class
class PromptBranch:
    """A branch in the PromptTree."""

    # Declare PromptBranch variables
    description = None  # Default description of the branch
    header = None       # Header prompt
    footer = None       # Footer prompt
    children = {}       # Dictionary of children branches with descriptions. Format={name: description (None for default)}
    initial_state = {}  # Initial state of the branch
    pass_info = {}      # Additional info to be passed to children
    insights = {}       # Dictionary of insights that the AI can update. Format={name: description}

    # Get branch ID
    @property
    def branch_id(self):
        """Get the branch ID."""
        return type(self).__name__

    def __init__(self, parent, **kwargs):
        """Initialize the PromptBranch."""
        self.parent = parent
        self.initialize_state(**kwargs)
        return

    def initialize_state(self, **kwargs):
        """Initialize the branch state."""
        # We allow kwargs to be passed in case the branch needs to be initialized with additional info
        self.state = {
            **self.initial_state,
            'insights': {x: None for x in self.insights.keys()}  # Initialize insights to None
        }
        return

    def __call__(self, messages):
        """Call the PromptBranch."""
        return (
            self.get_prompt(messages),
            self.get_tools(),
        )

    def get_pass_info(self):
        """Pass info to children."""
        return self.pass_info

    def get_prompt(self, messages):
        """Get the prompt."""

        # Initialze prompt
        prompt = []

        # Add preamble
        preamble = self.parent.preamble
        if preamble is not None:
            prompt.append(SystemMessage(content=preamble))

        # Add header
        header = self.get_header()
        if header is not None:
            prompt.append(SystemMessage(content=header))

        # Add messages
        prompt += messages

        # Add footer
        footer = self.get_footer()
        if footer is not None:
            prompt.append(SystemMessage(content=footer))

        # Add insights
        insights = self.get_insights()
        if insights is not None:
            prompt.append(SystemMessage(content=insights))

        # Return
        return prompt

    def get_header(self):
        """Get header."""
        return self.header

    def get_footer(self):
        """Get footer."""
        return self.footer

    def get_insights(self):
        """Get insights."""
        if len(self.insights) == 0:
            return None
        else:
            insights = f"Your insights so far are:"
            for name, state in self.state['insights'].items():
                insights += f"\n{name}: {state}"
            return insights

    def get_tools(self):
        """Get tools."""

        # Initialize tools
        tools = []

        # Add switch branch tool
        if len(self.children) > 0:
            tools.append(self._tool_switch_branch())

        # Add update insights tool
        if len(self.insights) > 0:
            tools.append(self._tool_update_insight())

        # Return
        return tools

    def _tool_switch_branch(self):
        """Create tool to select next branch."""

        # Get variables
        tool_name = SELECT_BRANCH
        children = self.children

        # Create tool function
        tool_func = self.switch_branch

        # Create tool description
        tool_description = "Select the next branch to continue the conversation. Your options are:"
        for branch_id, branch_description in children.items():
            if branch_description is None:
                branch_description = self.parent.all_branches[branch_id].description
            tool_description += f"\n{branch_id}: {branch_description}"

        # Create tool schema
        class ToolSchema(BaseModel):
            branch: str = Field(
                description="Select next branch.",
                enum=list(children.keys()),
            )

        # Create tool
        tool_obj = Tool(
            name=tool_name,
            func=tool_func,
            description=tool_description,
            args_schema=ToolSchema,
        )

        # Return
        return tool_obj

    def _tool_update_insight(self):
        """Create tool to update an insight."""

        # Get variables
        tool_name = UPDATE_INSIGHT
        insights = self.insights

        # Create tool function
        tool_func = self.update_insight

        # Create tool description
        tool_description = "Update an insight. You can choose to update any of the following insights:"
        for name, state in insights.items():
            tool_description += f"\n{name}: {state}"

        # Create tool schema
        class ToolSchema(BaseModel):
            insight: str = Field(
                description="Select insight to update.",
                enum=list(insights.keys()),
            )
            value: str = Field(
                description="New value of the insight.",
            )

        # Create tool
        tool_obj = Tool(
            name=tool_name,
            func=tool_func,
            description=tool_description,
            args_schema=ToolSchema,
        )

        # Return
        return tool_obj

    def switch_branch(self, branch):
        """Switch to a new branch."""

        # Switch parent tree branch
        self.parent.branch = self.parent.all_branches[branch](parent=self.parent, **self.get_pass_info())

        # Return function message
        message = FunctionMessage(
            name=SELECT_BRANCH,
            content=f"You have switched to the {branch} branch.",
            additional_kwargs={'internal_function': True},
        )
        return message

    def update_insight(self, insight, value):
        """Update an insight."""

        # Update insight
        self.state['insights'][insight] = value

        # Return function message
        message = FunctionMessage(
            name=UPDATE_INSIGHT,
            content=f"You have updated the {insight} insight to {value}.",
            additional_kwargs={'internal_function': True},
        )
        return message

### Define PromptTree ###

# Create PromptTree class
class PromptTree:
    """A decision tree for prompting the AI."""

    # Declare PromptTree variables
    preamble = None      # System prompt to put before each branch prompt
    first_branch = None  # Name of first branch to start the prompt tree
    all_branches = {}    # Dictionary of all branches in the tree. Format={branch_id: branch_class}

    def __init__(self):
        """Initialize the PromptTree branch state."""
        self.branch = self.all_branches[self.first_branch](parent=self)
        return

    def __call__(self, messages, **kwargs):
        """Call the PromptTree."""
        return self.branch(messages, **kwargs)

    def get_state(self):
        """Get the current branch state."""
        return {
            'branch_id': self.branch.branch_id,
            'branch_state': self.branch.state,
        }

    def load_state(self, state):
        """Load a branch from the state."""
        branch_id = state['branch_id']
        branch_state = state['branch_state']
        if branch_id not in self.all_branches:
            raise ValueError(f"Unknown branch_id: {branch_id}")
        self.branch = self.all_branches[branch_id](parent=self)
        self.branch.state = branch_state
        return

### Define TreeAgent ###

# Create TreeAgent class
class TreeAgent:
    """An AI agent based on the PromptTree class."""

    def __init__(self, tree, model):
        """Initialize the TreeAgent."""
        self.tree = tree
        self.model = model
        return

    def __call__(self, messages, **kwargs):
        """Call the TreeAgent."""
        return self.respond(messages, **kwargs)

    def get_state(self):
        """Get the current state of the TreeAgent."""
        return self.tree.get_state()

    def load_state(self, state):
        """Load the state of the TreeAgent."""
        self.tree.load_state(state)
        return

    def respond(self, messages):
        """Respond to the messages."""

        # Initialize new messages
        new_messages = []

        # Loop until no function calls
        while True:

            # Get the prompt
            prompt, tools = self.tree(messages+new_messages)

            # Get the response
            funcs = [format_tool_to_openai_function(t) for t in tools]
            response = self.model.predict_messages(prompt, functions=funcs)
            new_messages.append(response)

            # Check for function calls
            if 'function_call' in new_messages[-1].additional_kwargs:
                # Get function call
                func_call = new_messages[-1].additional_kwargs['function_call']
                func_name = func_call['name']
                func_args = ast.literal_eval(func_call['arguments'])
                func = [x.func for x in tools if x.name == func_name][0]

                # Call the function
                func_response = func(**func_args)
                new_messages.append(func_response)
                continue
            else:
                # If no function call, break
                break

        # Return
        return new_messages

####################################################################################################
####################################################################################################

### EXAMPLE ###

# Create PromptBranches
class BranchA(PromptBranch):
    header = "You love icecream, but you only like vanilla icecream."
    footer = "If you choose to respond make sure you mention icecream."
    description = "A Branch to talk about icecream."
    children = {
        'BranchB': 'If someone mentions anything fancy, be sure to switch to this branch.',
        'BranchC': None,
    }
class BranchB(PromptBranch):
    header = "You love fine wines, but only if they are over 10 years old."
    footer = "If you choose to respond make sure you mention wine."
    description = "A Branch to talk about wine."
    children = {
        'BranchA': None,
        'BranchC': None,
    }
class BranchC(PromptBranch):
    header = "You love going to the beach all the time no matter what."
    footer = "If you choose to respond make sure you mention that you love the beach."
    description = "A Branch to talk about the beach."
    children = {
        'BranchA': None,
        'BranchB': None,
    }

# Create PromptTree
class MyPromptTree(PromptTree):
    preamble = "You are an AI who is obsessed with a few things."
    first_branch = 'BranchA'
    all_branches = {
        'BranchA': BranchA,
        'BranchB': BranchB,
        'BranchC': BranchC,
    }

### CONVERSATION ###

# Initialize the AI
llm = ChatOpenAI(model="gpt-3.5-turbo-0613")
tree = MyPromptTree()
agent = TreeAgent(tree, llm)

# Create sample conversation
messages = []
while True:

    # Human input
    user_message = input("You: ")
    messages += [HumanMessage(content=user_message)]

    # AI response
    new_messages = agent(messages)
    for m in new_messages:
        print("AI:", m)
    messages += new_messages

While this may not be a perfect way to go about things, it does demonstrate that with a relatively small amount of code we can work with existing LangChain arcitecture to implement a toy model. I think that with a little bit of work this could be made into something very useful.

I would love to learn more about if/how I can help contribute to incorporate this.

[jsbryaniv]

@dosu-beta what is the process for integrating this into the core of LangChain? Is there a process for strategizing with a more experienced person?

I also think that this is more related to prompts than agents.

[dosubot[bot]]

🤖

Hello,

Thank you for your interest in adding a decision tree/flow chart-like prompt architecture to LangChain. This is a great idea and can indeed improve the chatbot workflows.

Based on your description, it seems like you want to add a new type of action that represents a decision node in your flow chart. This can be done by modifying the prompt.py file to include a new action type. Here's an example of how you might modify the FORMAT_INSTRUCTIONS string to include a new "Decision Node" action:

FORMAT_INSTRUCTIONS = """Use a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).

Valid "action" values: "Final Answer", "Decision Node" or {tool_names}

Provide only ONE action per $JSON_BLOB, as shown:

{{{{ "action": $TOOL_NAME, "action_input": $INPUT }}}}

For a "Decision Node", the action_input should be another json blob that specifies the condition and the actions for each outcome:

{{{{ "action": "Decision Node", "action_input": {{{{ "condition": $CONDITION, "true_action": $TRUE_ACTION, "false_action": $FALSE_ACTION }}}} }}}}

Follow this format:

Question: input question to answer
Thought: consider previous and subsequent steps
Action:

$JSON_BLOB

Observation: action result
... (repeat Thought/Action/Observation N times)
Thought: I know what to respond
Action:

{{{{ "action": "Final Answer", "action_input": "Final response to human" }}}}

In this example, $CONDITION is a string that represents the condition for the decision, and $TRUE_ACTION and $FALSE_ACTION are json blobs that represent the actions to take if the condition is true or false, respectively. These actions can be any valid action, including another "Decision Node".

Please note that you will also need to modify the code that processes the actions to handle the new "Decision Node" action type. This will likely involve writing a function that evaluates the condition and executes the appropriate action.

As for where this should belong in the object hierarchy, it would be best to place this new functionality within the prompt.py file, as it directly relates to how prompts are processed and handled.

I hope this helps! If you have any further questions or need more clarification, please don't hesitate to ask.

Best, Dosu

Sources

• libs/langchain/langchain/agents/structured_chat/prompt.py
• libs/langchain/langchain/evaluation/comparison/prompt.py
• libs/langchain/langchain/chains/qa_generation/prompt.py
• libs/langchain/langchain/evaluation/criteria/prompt.py
• libs/langchain/langchain/chains/llm_symbolic_math/prompt.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jsbryaniv]

@dosu-bot I meant is there a guideline for how to talk to a human more experienced than me about how I can contribute? I saw the first step is to open an issue, which is what I did here, but what is the next step? Do I just fork and push the proposed change? I would prefer to talk to someone ahead to time to make sure I do it right.

[dosubot[bot]]

🤖

Hello,

Thank you for your interest in contributing to LangChain. Your proposed feature sounds interesting and potentially very useful.

After opening an issue, the next step is indeed to fork the repository and start working on your proposed changes. However, it's understandable that you might want to discuss your ideas with a more experienced person before proceeding.

In the LangChain project, we encourage open discussion and collaboration. You can start a conversation about your proposed feature in the issue you've opened. This way, maintainers and other contributors can provide feedback and guidance. You can also directly contact a maintainer or send an email to hello@langchain.dev for more in-depth discussions.

When you're ready to submit your changes, please follow the "fork and pull request" workflow. Make sure your pull request follows the checked-in pull request template and passes the formatting, linting, and testing checks. If you're adding a feature, please also add a demo notebook in docs/modules and add unit and integration tests.

If you encounter any difficulties or have any questions about the process, don't hesitate to reach out. We're here to help and we appreciate your contribution to LangChain.

Best, Dosu

Sources

• .github/CONTRIBUTING.md
• docs/docs_skeleton/docs/community.md

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jsbryaniv]

@dosu-bot How do I find a "maintainer" to contact about my feature proposal?

[dosubot[bot]]

🤖

Hello,

Thank you for your interest in contributing to the LangChain project. Your feature proposal for a decision tree/flow chart-like prompt architecture sounds intriguing and could potentially add significant value to the project.

To discuss your feature proposal with someone experienced, you can directly contact the maintainers of the LangChain project. Based on the information provided, the current maintainers are:

1. @hwchase17 - project lead
2. @agola11 - responsible for Tracing / Callbacks and Async
3. @eyurtsev - responsible for DataLoader Abstractions

You can mention them in the issue you opened by using their GitHub usernames preceded by the "@" symbol. This will notify them about your message. You can also directly send them a message through GitHub.

Please make sure to provide as much detail as possible about your proposed feature, including the sample script you mentioned. This will help the maintainers understand your proposal better and provide more constructive feedback.

Remember, it's important to be patient as maintainers often have many responsibilities and it may take some time for them to respond.

I hope this helps and I look forward to seeing your feature proposal implemented in LangChain!

Best, Dosu

Sources

• .github/ISSUE_TEMPLATE/bug-report.yml

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[eyurtsev]

cross referencing: https://github.com/langchain-ai/langchain/discussions/10010

[jsbryaniv]

Hello, I wanted to know if this idea mentioned in this post would be something that would be helpful for LangChain, and if so what the steps would be to contribute to include something like this in the core of LangChain. I have never contributed to open source code before so I dont know how to get started. @aronweiler @howtodoml @eyurtsev @agola11 @hwchase17

[dosubot[bot]]

Hi, @jsbryaniv,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you opened this issue to request functionality for a decision tree/flow chart-like prompt architecture to be incorporated into LangChain. Detailed instructions on how to modify the prompt.py file to include the new feature have been provided, and collaboration has been encouraged. Additionally, it was suggested to contact the maintainers, including @hwchase17, @agola11, and @eyurtsev, to discuss the proposal.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation.

[melon-husk]

Hi, any update on this feature?

[tomdzh]

+1 any update on this? Looking forward to the integration.

[AumeshMisra]

Also looking forward to this. Any update here?

[jsbryaniv]

@AumeshMisra , @tomdzh , @melon-husk thanks for the interest in this feature! I have code to implement this functionality, but I have no experience with open source or integrating custom features into existing codebases. So right now I just have everything local on my machine. If any of you have experience with open source codebases and can help I would be willing to work together