firebase / genkit

An open source framework for building AI-powered apps with familiar code-centric patterns. Genkit makes it easy to develop, integrate, and test AI features with observability and evaluations. Genkit works with various models and platforms.
Apache License 2.0
756 stars 111 forks source link

[JS] Gemini JSON mode can't stream #958

Open mbleigh opened 1 month ago

mbleigh commented 1 month ago

Describe the bug When generating in JSON mode, the Gemini plugin crashes when trying to send partial responses. This is because we use standard JSON.parse instead of a lenient parser.

To Reproduce

const {stream} = generateStream({
  prompt: "Generate a list of 50 contacts for synthetic data testing.",
  output: {format: "json", schema: z.object({people: z.array(z.object({name: z.string(), email: z.string()}))})},
});

for await (const chunk of stream()) {
  // crashes here
  console.log(chunk);
}

Expected behavior A partial object is returned as a chunk.

cabljac commented 1 month ago

Still needs fixing in next