search

firebase / genkit

An open source framework for building AI-powered apps with familiar code-centric patterns. Genkit makes it easy to develop, integrate, and test AI features with observability and evaluations. Genkit works with various models and platforms.
Apache License 2.0
802 stars 121 forks source link

[Dev UI] Support arbitrary model configuration parameters #690

Open MichaelDoyle opened 4 months ago

MichaelDoyle commented 4 months ago

Currently we do not support model parameters that are not part of the common set. For the most part, this does not impact Gemini support outside of “safety” (see #689). It’s more noticeable with OpenAI and/or Anthropic, although most of these options are fairly niche.

This feature would dynamically render and bind form elements for any known configuration option.

Note: We also need to dynamically validate model configuration, which also applies to the standard configuration options. As an example, Open AI and Gemini have different ranges for temperature. This could be driven by the underlying schema definitions for each model.

mjchristy commented 1 week ago

Do we have examples of schema for extra features that I could reference? Do we think we could render UI based on the schema or is it safer to add a code block to the model runner?

MichaelDoyle commented 1 week ago

Sure, below are a bunch of examples.

I think my dream is to render based on the available options, and to even start driving validation from the underlying schema as well, even for the "stock" configurations. Right now, we set criteria for that (e.g. temp should be 0-1) in the UI code itself.

All that said, looking at some of these configuration options, it can get complicated pretty quickly. We might not want to show all of them. Maybe we need some flag/criteria for what gets rendered. There are some obvious missing ones that I think we need such as (1) safety (2) json mode (3) tool calling mode, etc.

There is also an alternative universe where we have a dotprompt style editor w/ autocomplete based on the schema.

Gemini Via Vertex:

https://github.com/firebase/genkit/blob/main/js/plugins/vertexai/src/gemini.ts#L53-L82

Open AI Compat: https://github.com/firebase/genkit/blob/main/js/plugins/vertexai/src/modelgarden/openai_compatibility.ts#L45-L53

Anthropic via Vertex https://github.com/firebase/genkit/blob/main/js/plugins/vertexai/src/modelgarden/anthropic.ts#L49-L51