OpenAI: Function calling and other API updates

[ryanpeach]

Feature request

https://openai.com/blog/function-calling-and-other-api-updates

I think we should update the ChatOpenAi models behavior with tools so that it used the native API.

Motivation

Their model is likely trained to handle functions this way, and will have a lot better support. It also supposedly guarantees json matching the json schema, which can be hard to achieve otherwise.

Your contribution

I may be able to help. I’m working on OpenAI at work, but I’m just learning langchains API.

[GameTaco]

Yep these look like really cool changes. It looks like it'll clean tools up a little, should increase the models understanding, and hopefully provide more dependable responses.

"models [new 0613 models] have been fine-tuned to both detect when a function should to be called (depending on the input) and to respond with JSON that adheres to the function signature."

The messages also have additional impact since the models have

"...more reliable steerability via the system message..."

and the the functions are implemented by being

"...injected into the system message in a syntax the model has been trained on."

That being said I think changes will need to be made in a few places. Informing the model about the tools looks mostly the same, but output parsers will need to know about the new finish response.

{
  "id": "chatcmpl-123",
  ...
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": null,
      "function_call": {
        "name": "get_current_weather",
        "arguments": "{ \"location\": \"Boston, MA\"}"
      }
    },
    "finish_reason": "function_call"
  }]
}

Prompts will need to know about the new function "role" too.

curl https://api.openai.com/v1/chat/completions -u :$OPENAI_API_KEY -H 'Content-Type: application/json' -d '{
  "model": "gpt-3.5-turbo-0613",
  "messages": [
    {"role": "user", "content": "What is the weather like in Boston?"},
    {"role": "assistant", "content": null, "function_call": {"name": "get_current_weather", "arguments": "{ \"location\": \"Boston, MA\"}"}},
    {"role": "function", "name": "get_current_weather", "content": "{\"temperature\": "22", \"unit\": \"celsius\", \"description\": \"Sunny\"}"}
  ],
  "functions": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city

A lot of the things I quoted are in the blog linked above, but some are from the new entry in the docs which also have examples in python. If anyone wants to work these changes in, I'll be willing to help. I don't have the time to implement it all in a reasonable time.

[Cdingram]

This is already implemented, see example here. One thing that does seem to be missing is function_call parameter that can be sent in the request (not the message response) as seen here. This allows you to specify how the model should call or not call specific functions or choose by itself.

[francisjervis]

I do not advise this. My experience is that it does not work reliably, at all. 1/4 calls do not return JSON, it ignores enums, and hallucinates "required" properties if they are not in the input. Should be considered harmful until it is more performant.

[ryanpeach]

This is already implemented, see example here. One thing that does seem to be missing is function_call parameter that can be sent in the request (not the message response) as seen here. This allows you to specify how the model should call or not call specific functions or choose by itself.

How did you do it so quickly? Wasn't it announced yesterday?

[GameTaco]

Ya, I barely had time to read about it and come here and post, and there were already example notebooks. Thanks to hwchase17 for being a coding ninja.

If anyone is looking for the second part that Cdingram was talking about, it's incorporated into an agent which also incorporates the tools to OpenAI Functions conversion method so you don't have to do that part separately.

[tracy-e]

Hi, I would like to know, can I use the function call feature in LLMChain?

conversation.run(input, functions=functions)

[GameTaco]

@tracy-e If you really need a chain, I found one that supports functions, but it looks like it's meant for extraction specifically. I got a successful result on this quick test though.

from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage
from langchain.tools.file_management import FileSearchTool
from langchain.chains.openai_functions import OpenAIFunctionsChain  
from langchain.prompts import PromptTemplate
from langchain.tools import format_tool_to_openai_function

tools = [FileSearchTool()]
functions = [format_tool_to_openai_function(t) for t in tools]

tools = [MoveFileTool()]
functions = [format_tool_to_openai_function(t) for t in tools]

llm = ChatOpenAI(temperature=.7, model="gpt-3.5-turbo-0613")
template = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title. There are local files that contain ideas for plays. You can use the following tools to access them:
file_search

Title: {title}
Playwright: This is a synopsis for the above play:"""
prompt_template = PromptTemplate(input_variables=["title"], template=template)
synopsis_chain = OpenAIFunctionsChain(llm=llm, prompt=prompt_template, output_key="synopsis", functions=functions)

input = "Title: The Tempest"
synopsis_chain.run(title=input) 

I'm not sure what you're trying to do, but there are other ways to use the functions feature. The docs are being worked on so they don't link to the example notebooks right now, but you can still find them in the repo.

This is the file with the FunctionsChain. There's no documentation on how to use it by itself, I just dug through it. The extraction functions (that uses the FunctionsChain) are documented here.

I'll add the examples for the implementations with examples below. The Agent is really the most full feature, because it can also utilize the received message. This is the Functions Agent, and this is the Functions Tools

[tracy-e]

@GameTaco Thank you very much for your reply. The information you provided is very useful to me. The extraction chain may meet my requirements. Alternatively, I may consider replacing the ConversationChain I am currently using with a Conversation Agent.