Guidance support - Githubissues

Guidance (https://github.com/microsoft/guidance) is a library that allows you to control LLM's output by using a DSL to guide the model toward the desired output or format. Here's how it could look like, using an example from Guidance's README paired with hypothetical ToT support:

import guidance
from tree_of_thoughts import OpenAILanguageModel
from tree_of_thoughts import MonteCarloTreeofThoughts

openai_api_model = "gpt-3.5-turbo"

guidance.llm = guidance.llms.OpenAI(openai_api_model)
tot_model = OpenAILanguageModel(api_key='api key', api_model=openai_api_model)
tree_of_thoughts = MonteCarloTreeofThoughts(tot_model)

# we can pre-define valid option sets
valid_weapons = ["sword", "axe", "mace", "spear", "bow", "crossbow"]

# define the prompt
program = guidance("""The following is a character profile for an RPG game in JSON format.
```json
{
    "description": "{{description}}",
    "name": "{{gen 'name'}}",
    "age": {{gen 'age' pattern='[0-9]+' stop=','}},
    "armor": "{{#select 'armor'}}leather{{or}}chainmail{{or}}plate{{/select}}",
    "weapon": "{{select 'weapon' options=valid_weapons}}",
    "class": "{{gen 'class'}}",
    "mantra": "{{gen 'mantra'}}",
    "strength": {{gen 'strength' pattern='[0-9]+' stop=','}},
    "items": [{{#geneach 'items' num_iterations=3}}
        "{{gen 'this'}}",{{/geneach}}
    ]
}```""")

# execute the prompt
response = tree_of_thoughts.solve(
    program(description="A quick and nimble fighter.", valid_weapons=valid_weapons)
    )

That's not the most elegant API, but it gives you a general idea of how those two libraries can significantly increase the quality and reproducibility of outputs. Guidance can be used for all kinds of prompts; please look at their documentation, as it should explain its uses and benefits much better than I could hope to do.

kyegomez / tree-of-thoughts

Guidance support #70