AshleyTuring
commented
5 months ago Thanks for writing this up @MichaelDoyle!
Just adding some comments here to perhaps save others some time. There are a few questions raised inline too!
The following is a sample dotPrompt saved as weather.prompt
You'll see at the top of the file I have used it with gpt-4o and gemini.
It would be great if we could pull the list of available models literal name strings (e.g. "googleai/gemini-1.5-pro-latest") via the genKit api, ideally, across all LLM providers i.e. Ollama, OpenAI, Google, Grok etc? If not this is not possible via the api, then having them listed in Github in one place would be a good start.
---
# model: googleai/gemini-1.5-pro-latest
model: openai/gpt-4o
config:
temperature: 0.6
# input:
# schema:
# type: object
# properties:
# cities:
# type: array
# items:
# type: string
# No 'required' keyword here, making it optional
output:
format: text
tools:
- getWeather
---
{{role "system"}}
Always try to be as efficient as possible, and request tool calls in batches.
{{role "user"}}
I really enjoy traveling to places where it's not too hot and not too cold.
{{role "model"}}
Sure, I can help you with that.
{{role "user"}}
Help me decide which is a better place to visit today based on the weather.
I want to be outside as much as possible. Here are the cities I am considering:
New York
London
Amsterdam
A small thing here to remember is to configureGenkit and not to forget the "apiVersion: 'v1beta'", otherwise, you'll get googleai/gemini-1.5-pro-latest not found.
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp();
import { dotprompt } from '@genkit-ai/dotprompt';
import { configureGenkit } from '@genkit-ai/core';
import { googleAI } from '@genkit-ai/googleai';
import { openAI } from 'genkitx-openai';
configureGenkit({ plugins: [
dotprompt(),
googleAI({ apiKey: '<YOUR_API_KEY>', apiVersion: 'v1beta' }),
openAI({ apiKey: '<YOUR_API_KEY>' })
] });
The calling code
const weatherPrompt = await prompt('weather');
promptResult = await weatherPrompt.generate({});I have tested this simple tool with Gemini and Gpt-4o
const getWeather = defineTool(
{
name: 'getWeather',
description: 'Get the weather for the given location.',
inputSchema: z.object({ city: z.string() }),
outputSchema: z.object({
temperatureF: z.number(),
conditions: z.string(),
}),
},
async (input) => {
const conditions = ['Sunny', 'Cloudy', 'Partially Cloudy', 'Raining'];
const c = Math.floor(Math.random() * conditions.length);
const temp = Math.floor(Math.random() * (120 - 32) + 32);
return {
temperatureF: temp,
conditions: conditions[c],
};
}
);
export { getWeather };The tool is called as expected, however, "promptResult.toolRequests" never seems to return anything, perhaps, I am not calling it correctly?
if (promptResult.toolRequests) {
for (const toolRequest of promptResult.toolRequests()) {
const tool = this._tools[toolRequest.name];
if (tool) {
await tool.run(this._agent_doc_ref, toolRequest?.input);
this._last_tool_completion_datetime = new Date(new Date().toUTCString());
await this.updateAgentData({
last_action: toolRequest?.name,
[`last_tool_completion_datetime_${this._agent_chat_id}`]: this._last_tool_completion_datetime,
[`current_prompt_step_${this._agent_chat_id}`]: this._current_prompt_step,
retry: retryCount
});
hasCalledTool = true;
}
}
}I have got around this by looking at the const responseHistory = promptResult.toHistory();
The toHistory returns messages with a content array of type toolRequest and toolResponse, so at least, we have access it to it.
content (array) (map)
toolRequest (map) input (map) city "New York" (string)
and a toolResponse
content (array) (map) toolResponse (map) name "getWeather" (string) output (map)I am wondering the best way to handle more complex tools let's say, whereabouts, we need to pass in parameters from the underlying code. For example, we might want to pass in the relevant IDatabase and associate database record key. As follows, you'll see I created a wrapper function for this called createToolWithContext:
const createToolWithContext = (database: IDatabase, agentDocId: string) => {
return <TInput, TOutput>(config: { name: string; description: string; inputSchema: z.ZodSchema<TInput>; outputSchema?: z.ZodSchema<TOutput>; }, handler: (input: TInput, context: { database: IDatabase; agentDocId: string }) => Promise<TOutput>) => {
return defineTool(config, (input: TInput) => handler(input, { database, agentDocId }));
};
};
/**
* Tool to save the user's first and last name.
*/
const saveName = (database: IDatabase, agentDocId: string) => createToolWithContext(database, agentDocId)(
{
name: 'SaveName',
description: 'Saves the user\'s first and last name.',
inputSchema: z.object({
firstName: z.string().optional(),
lastName: z.string().optional(),
}),
},
async (input, { database, agentDocId }) => {
// Split the agent document ID to get the user ID.
const userId = agentDocId.split('_')[0];
// Prepare the update data object.
const updateData: any = {};
if (input.firstName) {
updateData.firstName = input.firstName.trim();
}
if (input.lastName) {
updateData.lastName = input.lastName.trim();
}
// Update the user profile document in the database.
await database.set('userProfile', userId, updateData, true);
return { success: true };
}
);I assume the correct way is then to programmatically set it (when using dotPrompt), is this the correct approach?
dotPrompt.tools = [saveName(database, agentDocId)];
promptResult = await dotPrompt.generate({});Lastly, thanks for the notes on history. I may pass that into dotPrompt as a parameter and render it. It would be great if there was some way to auto summarise the history based on the number of tokens used and pass that in as history. The plumbing code is quite tedious to write and difficult to test. Is this something GenKit api could handle with the call to saveHistory()?
MichaelDoyle
mbleigh
ariel-pettyjohn
chrisraygill
kevinthecheung
Problem
The existence of both
definePromptanddefineDotPromptis causing confusion. See also: discussion https://github.com/firebase/genkit/discussions/337. I believe this at least partially stems from the following documentation which framesdefinePromptas a starting point, anddotpromptas a more advanced capability:https://firebase.google.com/docs/genkit/prompts
I don't think this is what we intend; rather I think we intend for developers to start with
dotprompt.Background history/context:
When we first implemented the dotprompt library, we had a method
definePromptthat was used to create and register a prompt action in the registry. Calling this action conveniently hydrated any input variables into the prompt and then called the model to generate a response.Later, when we added dotprompt functionality to the Developer UI, we needed an action that would simply render the prompt template without doing the generate step. This led to the following changes:
definePromptwas repurposed as a lower level method that registers a prompt action which returns aGenerateRequestwithout doing the generate step.defineDotPromptmethod, which followed the old semantics (render the template and then call generate). It usesdefinePromptunder the covers to register the prompt, which allows the Developer UI to call the underlying prompt action in cases where it only needs the rendered prompt template.Proposal
definePromptto something more indicative of what it should be used for.