pipecat-ai / pipecat

Open Source framework for voice and multimodal conversational AI
BSD 2-Clause "Simplified" License
3.08k stars 269 forks source link

from pipecat.processors.frameworks.langchain import LangchainProcessor,Usage of tools in langchain #298

Open geekofycoder opened 2 months ago

geekofycoder commented 2 months ago

I was trying to implement your interruptible langchain example by moulding it to my usecase.As we can call the tools in OpenAI,similarly I wanted to implement the tools with langchain.Is there a way to use the tools of Langchain in Pipecat?

TomTom101 commented 2 months ago

You sure can, the processor takes any Langchain Runnable. A chain with tools is basically this:

from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def multiply(first_int: int, second_int: int) -> int:
    """Multiply two integers together."""
    return first_int * second_int

llm = ChatOpenAI(model="gpt-4o").bind_tools([multiply])

Then use the llm in your chain

chain = prompt | llm

Does this help?

geekofycoder commented 2 months ago

yes it helps a lot and i think multiple agentic tools can be passed into the chain.Thanks for helping!

TomTom101 commented 2 months ago

Ideally you close the issue if your questions has been answered.

geekofycoder commented 2 months ago

openai.BadRequestError: Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_EuvlW1rYCMs6ne4mELZ3EWnB", 'type': 'invalid_request_error', 'param': 'messages.[5].role', 'code': None}}

This is the error that is coming after I bind the basic multiply and addition function to interruptible langchain example

geekofycoder commented 2 months ago

Ideally you close the issue if your questions has been answered.

The error occured,you can check in the comment thread

TomTom101 commented 2 months ago

Guess I should have tried it first :) I'll check and communicate here! Thanks for the heads-up!

geekofycoder commented 2 months ago

Guess I should have tried it first :) I'll check and communicate here! Thanks for the heads-up!

thats cool...guess the langchain and langgraph tool usage examples are also coming.....that will be really helpful

sthitap0 commented 2 months ago

@TomTom101 Any updates on the tool call? I am facing the same issue.

TomTom101 commented 1 month ago

I am still alive but running on little time currently. I hope to have some space next week to dive into this!