[Tooling] definePrompt defineDotPrompt design fault

[AshleyTuring]

Describe the bug

1. renderPrompt requires model which negates the point of defining it within definePrompt
2. defineDotPrompt is missing features

To Reproduce Steps to reproduce the behavior:

1. definePrompt(
   {
   name: 'helloGetNamePrompt',
   inputSchema: z.object({
     response: z.string(),
     retryCount: z.number().default(0),
     argTools: z.record(z.string(), z.any()),
     argHistory: z.array(z.record(z.string(), z.any())),
     argReturnToolRequests: z.boolean(),
   }),
   },
   async (input) => {
   const basePrompt = `Hello!?`;
   const retryPrompt = `We previously asked. You answered {{input.response}} Could you please provide your again?`;
   const promptText = input.retryCount > 0 ? retryPrompt : basePrompt;
   
   return {
     messages: [
       { role: 'system', content: [{ text: SYSTEM_PROMPT }] },
       { role: 'user', content: [{ text: promptText }] },
     ],
     //config: { model:'', temperature: 0.3 },
   
     history: input.argHistory,
     tools: Object.values(input.argTools),
     returnToolRequests: input.argReturnToolRequests,
     model: gpt4o, // << MODEL DEFINITION SUPPORTED
   };
   }
   );

2. Try to use the prompt defined in 1

const renderedPrompt = await renderPrompt({ 
      prompt: definePromptDefinedAbove, 
      input: { 
          response: "hello", 
          retryCount: 0, 
          argTools: Object.values(this._tools), 
          argHistory: history, 
          argReturnToolRequests: true 
      } ,
      model: 'googleai/gemini-pro' SHOULD NOT BE REQUIRED // << model is required
  });

As you can see the model is required. This negates the point of defining it in the definePrompt

5. Furthermore defineDotPrompt does not supprot tools, history, context so it cannot be used for advanced prompting which makes it limited

Expected behavior Model should not be required in renderPrompt

[MichaelDoyle]

Hey Ashley - thanks for the report. Sorry this is confusing. :)

The title mentions defineDotPrompt but then in your code you are referencing definePrompt and renderPrompt. Can you clarify?

definePrompt and renderPrompt are part of the framework code, and are intended for plugin developers to write libraries that implement prompt behavior (e.g. an alternative to dotprompt...)

If you intend to make use of dotprompt, you would want to use defineDotPrompt instead. Here are some examples in the docs: https://firebase.google.com/docs/genkit/dotprompt.

Essentially you'll be able to do something like the following:

const prompt = defineDotPrompt(...);
prompt.generate(input)

You'll have access to everything you need, including tools, history, etc.

[AshleyTuring]

Thanks for the help Michael, sorry about the confusion. To clarify I have to use

definePrompt

As I am using tools, context and history (which bizarrely have to be passed in to just be passed out, see argReturnToolRequests, argHistory etc).

Anyhow, that aside, "definePrompt" allows for the "model:" (property) to be defined, however, as far as I can tell, it is pointless because "model:" is required when using "renderPrompt". Thus the design of "renderPrompt" appears flawed as there are no other helpers to use when calling the "generate" method with a "definePrompt" prompt.

Either a new helper, like "renderPrompt", is needed or "model:" needs to be optional on the "renderPrompt" method.

Furthermore, there does not appear to be any immediate alternative approach because "defineDotPrompt" does not support tools, context or history. I'd love to use defineDotPrompt but it lacks key functionality.

Unfortunately, the entire prompt framework both "definePrompt" (which mixes the code definition with model) and "defineDotPrompt" (which has benefits of separation of code but cannot be used as it does not support tools, context or history), especially, the naming convention between both may also need rethinking in terms of functionality - it seems confused and perhaps requires more thought.

For example: the defineDotPrompt should support config, tools, context and history that way it could be used without having to resort to definePrompt.

definePrompt could be called definePluginPrompt so its intention is clearer.

[MichaelDoyle]

No worries, I mean to say, I am sorry you had a confusing experience getting started with Genkit. defineDotPrompt supports all of the things you mention. Here is an example a flow using defineDotPrompt that uses tools and history below.

The main thing to understand is that dotprompt is 2 parts; (1) configuration parameters and (2) a prompt template. The configuration is where things like model, temperature, and tool references go. The prompt template drives the creation of messages, which includes things like history. The most natural fit for this is when storing prompts as their own separate text files (.prompt files), and then loading them in via prompt(), but you can of course define them in code as well using defineDotPrompt, as the code below does.

import { defineTool } from '@genkit-ai/ai/tool';
import { defineDotprompt } from '@genkit-ai/dotprompt';
import { defineFlow } from '@genkit-ai/flow';
import { gemini15Flash } from '@genkit-ai/googleai';
import * as z from 'zod';

const getWeather = defineTool(
  {
    name: 'getWeather',
    description: 'Get the weather for the given location.',
    inputSchema: z.object({ city: z.string() }),
    outputSchema: z.object({
      temperatureF: z.number(),
      conditions: z.string(),
    }),
  },
  async (input) => {
    const conditions = ['Sunny', 'Cloudy', 'Partially Cloudy', 'Raining'];
    const c = Math.floor(Math.random() * conditions.length);
    const temp = Math.floor(Math.random() * (120 - 32) + 32);

    return {
      temperatureF: temp,
      conditions: conditions[c],
    };
  }
);

const template = `
  {{role "system"}}
  Always try to be as efficient as possible, and request tool calls in batches.

  {{role "user"}}
  I really enjoy traveling to places where it's not too hot and not too cold. 

  {{role "model"}}
  Sure, I can help you with that. 

  {{role "user"}}
  Help me decide which is a better place to visit today based on the weather. 
  I want to be outside as much as possible. Here are the cities I am 
  considering:\n\n{{#each cities}}{{this}}\n{{/each}}`;

export const weatherPrompt = defineDotprompt(
  {
    name: 'weatherPrompt',
    model: gemini15Flash,
    input: {
      schema: z.object({
        cities: z.array(z.string()),
      }),
    },
    output: {
      format: 'text',
    },
    config: {
      maxOutputTokens: 2048,
      temperature: 0.6,
      topK: 16,
      topP: 0.95,
    },
    tools: [getWeather],
  },
  template
);

defineFlow(
  {
    name: 'flowWeather',
    inputSchema: z.object({
      cities: z.array(z.string()),
     }),
    outputSchema: z.string(),
  },
  async (input) => {
    const response = await weatherPrompt.generate({
      input,
    });

    return response.text();
  }
);

[AshleyTuring]

That's great thank you Michael, I'll try it tomorrow. From the quick look I didn't see History, would that have to be dynamically added to the template variable somehow?

Also would you be able to provide the example in both .prompt and code formats?