Trying to use tools leads to duplication or confusion for ai runner

[colelawrence]

I had the following code in my codebase:

return createProxyForPromiseMethods(async () => ({
  run: createAIRunner({
    chatModel: await chatModel({
      model: options.model ?? "gpt-3.5-turbo",
      response_format: { type: "json_object" },
      tools: allTools.map((t) => ({
        type: "function",
        function: t.spec,
      })),
      tool_choice: match(options.tools)
        .ONLY((fn): Model.Chat.Config["tool_choice"] | undefined => ({
          type: "function" as const,
          function: { name: fn.spec.name },
        }))
        ._(() => undefined),
    }),
    functions: allTools,
    functionCallConcurrency: options.functionCallConcurrency,
    mode: "tools",
  }),
}));

Unfortunately, the way which dexter merges the options from ChatModel with "AI Runner", leads to there being both functions + tools specified, which is what I believe leads to instances where Open AI is being sent double function specs for this kind of code above.

I suspect this happens near chatModel.run in a call to const mergedParams = deepMerge(this.params, params); which ends up duplicating the entries for the list of functions. Sometimes I also get an error that looks like several functions in this list without names (name = "", description = ""). I wonder if it is related...

[rileytomasek]

@colelawrence hmm I don't think anyone has tried supplying tools to both the ChatModel and createAIFuntion() args. I also haven't seen response_format: { type: "json_object" } used in combination with tools — has that been working for you elsewhere?

The merging code you linked to appears to be the issue because I doubt it will be smart enough to dedupe the tool objects when merging the params, so you will end up with dupes when merging objects with arrays of objects.

We could try and improve the deep merging of params to look inside objects in arrays. A temporary workaround would be to just set tools on the ChatModel or createAIFuntion() args. Am I understanding your issue properly?

[colelawrence]

@rileytomasek - yeah, I'm not really sure what the combinations are. So far the config I had has been working, even though I am not allowing the model to respond at all. I will improve my configuration with your tips. Here's another deepMerge related issue, but this time with AbortSignal - this leads to Ky hitting an error.