Iterate on syntax of expressing nodes and wires in JS

[PaulKinlan]

I was thinking about the graph viewer and it got me to thinking about the inverse, a graph creator.

If I was building a UI to create a graph, how would I know what the expected in and out values of each node are? I can't see an easy way to know how to build the edges in the graph without the user knowing directly that "in" is text and "out" is completion (for the 'completion' node connecting to the 'output' node)

[dglazkov]

OMG yes. The current API surface of a node is a matter of documentation. I wonder if there's a "Design view" of a node that also declares ins/outs.

[seefeldb]

This will also be useful for automatic graph generation and feedback cycles for the LLM. Essentially type checking the graph.

But I want to propose something a little cheeky in the meantime: "learn" from examples what can connect to what and how and see how far that gets us. This might be as simple as feeding the LLM the current graph + examples of other uses of the current node type (maybe biased towards graphs that are otherwise similar, once we have so many that this is an issue) and asking it what else it would connect and how and suggesting that to the user. I've seen glimpses of that in Visual Code with Copilot when writing breadboards.

I think this could be an interesting building experience. And it's a baby step towards graph generation.

[dglazkov]

This seems important in the context of more autocomplete friendly syntax for wiring.

[dglazkov]

Some ideas:

// ---- CURRENT
var template = kit.promptTemplate(...)
var input = board.input();
input.wire(
  "say->topic",
  board.include(NEWS_BOARD_URL).wire(
    "headlines->",
    template.wire("topic<-say", input).wire(
      "prompt->text",
      kit
        .generateText()
        .wire("<-PALM_KEY.", kit.secrets(["PALM_KEY"]))
        .wire("completion->say", board.output())
    )
  )
);

// ---- OPT 2 - explicit nodes ----------
var template = kit.promptTemplate(...)
var llm = kit.generateText()
  .wire("<-PALM_KEY.", kit.secrets(["PALM_KEY"]))
  .wire("completion->say", board.output())
  .wire("text<-prompt", template)
var news = board.include(NEWS_BOARD_URL)
  .wire("headlines->", template);
var input = board.input()
  .wire("say->topic", news)
  .wire("say->topic", template);

// ---- OPT 3 - define nodes upfront, wire props, not nodes --------------
var template = kit.promptTemplate(...)
let input = board.input();
let output = board.output();
let llm = kit.generateText();

llm.$PALM_KEY.wireIn(kit.secrets(["PALM_KEY"].$PALM_KEY));
llm.$text.wireIn(template.$prompt);
llm.$completion.wire(output.$say);

let news = board.include(NEWS_BOARD_URL);
input.$say
    .wire(news.$topic)
    .wire(template.$topic);
news.$headlines.wire(template.$headlines);
template.$prompt.wire(llm.$text);

[dglazkov]

Building on OPT 3, a way to address dynamic (like in a template):

template.$("headlines").wire(...)

[dglazkov]

Another potential benefit to wiring properties, rather than nodes: we can differentiate between inputs and outputs. You can wire from output property, but only to input property, and vice versa.

[dglazkov]

Seems reasonable to add supplemental type information for variable input/output nodes.

type NewsTemplate = {
  headlines: string;
  topic: string;
}

type Input = {
  say: string;
}

const template = kit.promptTemplate<NewsTemplate>(...)
const input = board.input<Input>();

[romannurik]

FWIW I think it's really useful (both conceptually and for type-safety if using TypeScript) to separate inputs and outputs, e.g.:

type NewsInput = { topic: string; }
type NewsOutput = { headlines: string; }
const template = kit.promptTemplate<NewsInput, NewsOutput>(...)

As a fun side note, in this model, board.input as an input terminal only has outputs but no inputs, and symmetrically, board.output as an output terminal only has inputs but no outputs. 🙃

[romannurik]

Two more quick thoughts:

1) Bonus points for syntaxes that are even more declarative so you could build a graph visualizer without having to do static analysis or run the program and export :)

2) Something to consider is encoding optional/required inputs somehow, so your runtime "incomplete graph" errors become compile-time errors:

type NewsInput = {
  topic: string;
  requiredParam: string;
  optionalParam?: string;
}

const template = kit.promptTemplate<NewsInput, NewsOutput>({
  topic: board.input().$say
  // compile error: missing requiredParam
}, ...);

And the library could offer a helper type Wires<T> that converts the user's simple input type { foo: string; } to a wire-able type { foo: InputWire<string>; } or such to use in constructors

[dglazkov]

Since the default use case is to wire forward (from input to input), plain property names as inputs:

kit.promptTemplate().template.wire(...)

And dollar-sign as outputs:

kit.promptTemplate().$prompt.wireIn(...)

[seefeldb]

Why not fail the type check when wiring out from an input, because it lacks the .wire property?

I liked the $ prefix for inputs and outputs to distinguish from wire etc al, otherwise we can't call an output wire.

[dglazkov]

I liked the $ prefix for inputs and outputs to distinguish from wire etc al, otherwise we can't call an output wire.

Oh I was thinking we'll ditch wire altogether from BreadboardNode.

[seefeldb]

Ah! How do we do optional and const wires? And how does specifying the input in a regular (->) wire look like?

[seefeldb]

We also have to pass the destination node as a function parameter (as it's a local variable, not a property on an object) and I think it feels semantically wrong for outgoing data to be a parameter of a call on a property: that's how I would do an incoming wire, actually. So for outgoing I'd still do a .wire or .to or so, no?

[dglazkov]

Let me try another idea out, taking simplest.json as example:

Current:

const completion = kit.generateText();
kit.secrets(["PALM_KEY"]).wire("PALM_KEY", completion);
simplest
  .input()
  .wire("text", completion.wire("completion->text", simplest.output()));

New:

import type { LLMStarterSecrets } from "@google-labs/llm-starter";

type Input = {
  text: string;
}

type Output = {
  text: string; 
}

const completion = kit.generateText();
kit.secrets<LLMStarterSecrets>(["PALM_KEY"]).PALM_KEY(
  completion.$PALM_KEY);
simplest
  .input<SimplestInput>().text(
    completion.$text).completion(simplest.output<Output>().$text);

[dglazkov]

is it just me, or is it even harder to read?

[dglazkov]

another idea. Use basically the same wire syntax, just break apart the input and output:

simplest
  .input<Input>()
  .wire("text->", completion.wire("completion->", "text", simplest.output()));

We can then type guard with string literals & unions. Like:

simplest
  .input<Input>()
  .wire("somethingOtherThanText->", ...);

Will throw an error, because the only allowed value for wire will be text->

WDYT?

[dglazkov]

Here's how it would be declared:

interface Input {
  text: string;
  data: string;
}

type WireSpec<Type> = {
  wire(spec: `${string & keyof Type}->`): void;
}

declare function input<Type>(): WireSpec<Type>;

input<Input>().wire("text->");
input<Input>().wire("data->");

input<Input>().wire("foo->"); // a TS compile error.

See this TS playground

[dglazkov]

Okay, we may not even need to split the spec:

interface Input {
  text: string;
  data: string;
}

interface Prompt {
  prompt: string;
}

type WireSpec<From> = {
  wire<To>(spec: `${string & keyof From}->${string & keyof To}` | `${string & keyof From}->`, to: To): To;
}

declare function input(): Input & WireSpec<Input>;

declare function template(): Prompt & WireSpec<Prompt>;

input().wire("text->", template());
input().wire("foo->", template()); // TS compile error
input().wire("text->prompt", template());
input().wire("text->promprr", template()); // TS compile error

See on TS playground

[dglazkov]

Also get intellisense

[dglazkov]

okay, I have a path to making nodes strongly typed by just using the bits from comment.

I am going to try landing this. Should be a fairly small change to the overall surface.

Please yell with loud objections.

[dglazkov]

Teaching it to properly recognize shorthand and longhand wire spec:

interface Input {
  text: string;
  data: string;
}

interface Prompt {
  text: string;
  prompt: string;
}

type Common<From, To> = Pick<To, keyof From & keyof To>;

type LongSpec<From, To> = `${string & keyof From}->${string & keyof To}`;

type ShortSpec<From, To> = `${string & keyof Common<From, To>}->`;

type WireSpec<From> = {
  wire<To>(spec: LongSpec<From, To> | ShortSpec<From, To>, to: To): To;
}

declare function input(): Input & WireSpec<Input>;

declare function template(): Prompt & WireSpec<Prompt>;

input().wire("text->", template());
input().wire("data->", template()); // TS compile error
input().wire("foo->", template()); // TS compile error
input().wire("text->prompt", template());
input().wire("text->promprr", template()); // TS compile error

In TS Playground

[dglazkov]

Playing with this now, and it's both nice (compile errors!) and terrible: error messages are just bad and intellisense only works half-the time. 😢

[benmathes]

OMG yes. The current API surface of a node is a matter of documentation. I wonder if there's a "Design view" of a node that also declares ins/outs.

FWIW, having a "design view" sorta feels like a code smell: That the code API to construct the nodes isn't self-explanatory?

E.g. this pseudocode where the structure is sorta in-the-code?

const board = new Board();
const output = board.output();
const askUserNode = board.input();
const kit = board.addKit(ChatAgent)
const translateToJapaneseNode = kit.chatWith("can you translate this to japanese for me?");

askUserNode
  .wireTo(kit.ChatWith("please translate this to Japanese:")
  .wireTo(kit.ChatWith("please make this very formal")
  .wireTo(board.output(); 

Admittedly text indentation can only ever be a tree, not a DAG

[dglazkov]

There are two different things conflated in this bug. One is the syntax and the other is about node inputs and outputs being declared. Let me split off the inputs/outputs bits into a separate bug.