Guidance sends the messages as if they were just plain text messages with a TypeScript code snippet:
{
"role": "assistant",
"content": "\n"
"```typescript\n"
"functions.get_current_weather({\n"
' "location": "New York City"\n'
"})```",
},
The OpenAI example in their documentation as well as the OpenAI cookbook example sends the messages back using the same format as they're returned from the API. I believe Guidance should also use this format, as that's probably what the model is fine tuned for.
Example from OpenAI cookbook (note the messages with 'content': None, 'function_call': {'arguments': ..., 'name': ...):
[{'content': 'Answer user questions by generating SQL queries against the '
'Chinook Music Database.',
'role': 'system'},
{'content': 'Hi, who are the top 5 artists by number of tracks?',
'role': 'user'},
{'content': None,
'function_call': {'arguments': '{\n'
' "query": "SELECT art.Name AS Artist, '
'COUNT(trk.TrackId) AS NumOfTracks FROM '
'Artist art INNER JOIN Album alb ON '
'art.ArtistId = alb.ArtistId INNER JOIN Track '
'trk ON alb.AlbumId = trk.AlbumId GROUP BY '
'art.Name ORDER BY NumOfTracks DESC LIMIT '
'5;"\n'
'}',
'name': 'ask_database'},
'role': 'assistant'},
{'content': "[('Iron Maiden', 213), ('U2', 135), ('Led Zeppelin', 114), "
"('Metallica', 112), ('Lost', 92)]",
'name': 'ask_database',
'role': 'function'},
{'content': 'What is the name of the album with the most tracks?',
'role': 'user'},
]
To Reproduce
Run the tool use notebook with this code for logging to observe the requests sent to OpenAI chat completions API:
The bug
When an assistant message with an intent to call a function (
function_call
is set) is sent back to the OpenAI Chat Completions API, it doesn't follow the format OpenAI uses in their examples Function calling and "OpenAI cookbooks – How to call functions with chat models".Guidance sends the messages as if they were just plain text messages with a TypeScript code snippet:
The OpenAI example in their documentation as well as the OpenAI cookbook example sends the messages back using the same format as they're returned from the API. I believe Guidance should also use this format, as that's probably what the model is fine tuned for.
Example from OpenAI cookbook (note the messages with
'content': None, 'function_call': {'arguments': ..., 'name': ...
):To Reproduce
Run the tool use notebook with this code for logging to observe the requests sent to OpenAI chat completions API:
System info (please complete the following information):
guidance.__version__
): 0.0.64