Closed pavelgj closed 2 months ago
Right now when you stream output from a generate call in JSON mode you still get raw model output that is hard to work with.
generate
Ex:
const GameCharactersSchema = z.object({ characters: z .array( z .object({ name: z.string().describe('Name of a character'), abilities: z .array(z.string()) .describe('Various abilities (strength, magic, archery, etc.)'), }) .describe('Game character') ) .describe('Characters'), }); const response = await generate({ model: gemini15Flash, output: { schema: GameCharactersSchema, }, prompt: `Respond as JSON only. Generate ${count} different RPG game characters.`, streamingCallback, }); return response.text();
Output
{"index":0,"content":[{"text":"```"}]} {"index":0,"content":[{"text":"json\n{\n \"characters\": [\n {\n \"name"}]} {"index":0,"content":[{"text":"\": \"Anya the Swift\",\n \"abilities\": [\n \""}]} {"index":0,"content":[{"text":"Stealth\",\n \"Swordsmanship\",\n \"Acrobatics\",\n \"Lockpicking\"\n ],\n \"background\": \"A"}]} {"index":0,"content":[{"text":" cunning thief who roams the shadows, seeking adventure and riches.\",\n \"alignment\": \"Chaotic Good\"\n },\n {\n \""}]} {"index":0,"content":[{"text":"name\": \"Grimbold the Stout\",\n \"abilities\": [\n \"Strength\",\n \"Axe Mastery\",\n \"Endurance\",\n \"Intimidation\"\n ],\n \"background\": \"A"}]} {"index":0,"content":[{"text":" seasoned warrior with a heart of gold, who fights for what he believes is right.\",\n \"alignment\": \"Lawful Good\"\n },\n {\n \"name\": \"Elara the Wise\",\n \"abilities"}]} {"index":0,"content":[{"text":"\": [\n \"Arcane Magic\",\n \"Healing\",\n \"Divination\",\n \"Lore\"\n ],\n \"background\": \"A powerful sorceress seeking knowledge and understanding of the world.\",\n \"alignment\": \"Neutral Good\"\n }\n ]\n}\n```"}]} {"index":0,"content":[{"text":""}]} {"name":"93c286e9-d5be-4e1c-81df-06af117ad125","done":true,"result":{"response":"```json\n{\n \"characters\": [\n {\n \"name\": \"Anya the Swift\",\n \"abilities\": [\n \"Stealth\",\n \"Swordsmanship\",\n \"Acrobatics\",\n \"Lockpicking\"\n ],\n \"background\": \"A cunning thief who roams the shadows, seeking adventure and riches.\",\n \"alignment\": \"Chaotic Good\"\n },\n {\n \"name\": \"Grimbold the Stout\",\n \"abilities\": [\n \"Strength\",\n \"Axe Mastery\",\n \"Endurance\",\n \"Intimidation\"\n ],\n \"background\": \"A seasoned warrior with a heart of gold, who fights for what he believes is right.\",\n \"alignment\": \"Lawful Good\"\n },\n {\n \"name\": \"Elara the Wise\",\n \"abilities\": [\n \"Arcane Magic\",\n \"Healing\",\n \"Divination\",\n \"Lore\"\n ],\n \"background\": \"A powerful sorceress seeking knowledge and understanding of the world.\",\n \"alignment\": \"Neutral Good\"\n }\n ]\n}\n```"}}
It would be much better if generate returned partial JSON, like this:
{"characters":[{"name":"Aella Stormrider"}]} {"characters":[{"name":"Aella Stormrider","abilities":["Lightning Magic","Swordsmanship","Eagle Vision"]}]} {"characters":[{"name":"Aella Stormrider","abilities":["Lightning Magic","Swordsmanship","Eagle Vision"]},{"name":"Torvin Stonefist","abilities":["Blacksmithing","Heavy Armor","Two-Handed Weapons"]},{}]} {"characters":[{"name":"Aella Stormrider","abilities":["Lightning Magic","Swordsmanship","Eagle Vision"]},{"name":"Torvin Stonefist","abilities":["Blacksmithing","Heavy Armor","Two-Handed Weapons"]},{"name":"Lyra Whisperwind","abilities":["Stealth","Lockpicking","Poison Craft"]}]} {"characters":[{"name":"Aella Stormrider","abilities":["Lightning Magic","Swordsmanship","Eagle Vision"]},{"name":"Torvin Stonefist","abilities":["Blacksmithing","Heavy Armor","Two-Handed Weapons"]},{"name":"Lyra Whisperwind","abilities":["Stealth","Lockpicking","Poison Craft"]}]} {"name":"76efd20d-53bc-4364-91ac-7ffead5aed3c","done":true,"result":{"response":"```json\n{\"characters\": [{\"name\": \"Aella Stormrider\", \"abilities\": [\"Lightning Magic\", \"Swordsmanship\", \"Eagle Vision\"]}, {\"name\": \"Torvin Stonefist\", \"abilities\": [\"Blacksmithing\", \"Heavy Armor\", \"Two-Handed Weapons\"]}, {\"name\": \"Lyra Whisperwind\", \"abilities\": [\"Stealth\", \"Lockpicking\", \"Poison Craft\"]}]}\n```"}}
I've opened a draft PR here, let me know what you think of this approach?
https://github.com/firebase/genkit/pull/484/files
https://github.com/firebase/genkit/pull/484 is not merged yet, right?
Sorry, just saw this, not yet no
Right now when you stream output from a
generate
call in JSON mode you still get raw model output that is hard to work with.Ex:
Output
It would be much better if
generate
returned partial JSON, like this: