Incorrect formatting of OpenAI function calls in previously completed messages

[fson]

The bug

When an assistant message with an intent to call a function (function_call is set) is sent back to the OpenAI Chat Completions API, it doesn't follow the format OpenAI uses in their examples Function calling and "OpenAI cookbooks – How to call functions with chat models".

Guidance sends the messages as if they were just plain text messages with a TypeScript code snippet:

{
    "role": "assistant",
    "content": "\n"
    "```typescript\n"
    "functions.get_current_weather({\n"
    '  "location": "New York City"\n'
    "})```",
},

The OpenAI example in their documentation as well as the OpenAI cookbook example sends the messages back using the same format as they're returned from the API. I believe Guidance should also use this format, as that's probably what the model is fine tuned for.

Example from OpenAI cookbook (note the messages with 'content': None, 'function_call': {'arguments': ..., 'name': ...):

[{'content': 'Answer user questions by generating SQL queries against the '
             'Chinook Music Database.',
  'role': 'system'},
 {'content': 'Hi, who are the top 5 artists by number of tracks?',
  'role': 'user'},
 {'content': None,
  'function_call': {'arguments': '{\n'
                                 '  "query": "SELECT art.Name AS Artist, '
                                 'COUNT(trk.TrackId) AS NumOfTracks FROM '
                                 'Artist art INNER JOIN Album alb ON '
                                 'art.ArtistId = alb.ArtistId INNER JOIN Track '
                                 'trk ON alb.AlbumId = trk.AlbumId GROUP BY '
                                 'art.Name ORDER BY NumOfTracks DESC LIMIT '
                                 '5;"\n'
                                 '}',
                    'name': 'ask_database'},
  'role': 'assistant'},
 {'content': "[('Iron Maiden', 213), ('U2', 135), ('Led Zeppelin', 114), "
             "('Metallica', 112), ('Lost', 92)]",
  'name': 'ask_database',
  'role': 'function'},
 {'content': 'What is the name of the album with the most tracks?',
  'role': 'user'},
]

To Reproduce

Run the tool use notebook with this code for logging to observe the requests sent to OpenAI chat completions API:

import logging
logging.basicConfig(filename="/workspaces/guidance/tool_use.log", encoding="utf-8", level=logging.DEBUG)
logging.getLogger("openai").setLevel(logging.DEBUG)

System info (please complete the following information):

• OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): N/A (GitHub Codespaces)
• Guidance Version (guidance.__version__): 0.0.64

[nc]

@slundberg thoughts? I think this might cause performance issues.

[willswordh]

Any update from the dev team?