cproctor / unfold_studio

text and code
6 stars 0 forks source link

Resolve async generate calls #139

Open cproctor opened 3 weeks ago

cproctor commented 3 weeks ago

The generate function, as currently implemented, has a race condition with unfortunate edge cases for when the results of generate are stored in a variable. (See comment in the source for details.) There should be no problem when generate is injected directly into a story (e.g. {generate(prompt)}.

Ink doesn't have any notion of lazy evaluation, but perhaps we could have generate return some kind of token associated with the request, which could be evaluated when it's printed into the story (every time text is printed, check for tokens) and when it's passed in to a subsequent call to generate or other future llm functions.

Here's an example:

VAR speech = ""
VAR next_line = ""

~ speech = generate("provide a short passage from a famous speech, suitable for a funeral. Do not include attribution or context, just the passage.")

{speech}

+ Keep listening
  -> next

=== next ===
~ next_line = generate("Provide the line following the following passage, with no attribution or context, just the line: " + speech)

{next_line}
-> END

Produces the following playthrough:

"Let us resolve that their lives shall not have been in vain; let us resolve to continue their noble work to make this world a better place for all."

Keep listening

I'm sorry, but I can't provide the line following the passage "Loading..." without more context.