firebase / genkit

An open source framework for building AI-powered apps with familiar code-centric patterns. Genkit makes it easy to develop, integrate, and test AI features with observability and evaluations. Genkit works with various models and platforms.
Apache License 2.0
773 stars 115 forks source link

[JS] need to standardize void tool input handlng across model plugins #1361

Open pavelgj opened 1 day ago

pavelgj commented 1 day ago

Context: https://discord.com/channels/1255578482214305893/1309139828310540369

Here's a simple repro for OpenAI:

import { genkit, z } from 'genkit';
import { gpt4o, openAI } from 'genkitx-openai';

const ai = genkit({
  plugins: [openAI()],
  model: gpt4o,
});

const gablorken = ai.defineTool(
  {
    name: 'gablorken',
    description: 'use to calculate a gablorken',
  },
  async () => {
    return 11;
  }
);

const helloFlow = ai.defineFlow('hello', async () => {
  const { text } = await ai.generate({
    prompt: 'what is a gablorken',
    tools: [gablorken],
  });
  return text;
});

(async () => {
  console.log(await helloFlow());
})();

Throws:

  error: {
    message: `Invalid schema for function 'gablorken': schema must be a JSON Schema of 'type: "object"', got 'type: "None"'.`,
    type: 'invalid_request_error',
    param: 'tools[0].function.parameters',
    code: 'invalid_function_parameters'
  },

This works fine for GoogleAI/VertexAI/

a workaround:

const gablorken = ai.defineTool(
  {
    name: 'gablorken',
    description: 'use to calculate a gablorken',
    inputSchema: z.object({}), // <------------
  },
  async () => {
    return 11;
  }
);
pavelgj commented 1 day ago

/cc @mbleigh

mbleigh commented 22 hours ago

I think we have two options here:

  1. Translate "undefined input schema" to {type: "object", additionalProperties: false} when converting to a model request
  2. Leave it up to the plugins to do same

I'm on the fence - it seems totally reasonable to me to have a tool with no arguments, but an empty object is probably effectively the same and will fix this problem for everyone...

Probably 2?