Silent generation

[mlashcorp]

Is your feature request related to a problem? Please describe. I've used guidance before the current refactor. We used to be able to call a guidance program with silent=True to avoid having any rendering to stdout. I'm currently unable to do this. The closest I found was to use with silent(): context but I still get the generation preview on stdout.

Describe the solution you'd like A way to have a fully silent generation

Describe alternatives you've considered I've tried sending silent=True as a kwarg to the gen() function

Additional context Example code I tried:

import os
from guidance import models, gen, user, assistant, system, silent

gpt = models.OpenAI("gpt-3.5-turbo-1106", api_key=os.getenv("OPENAI_KEY"))

def ask_question(corpus, user_question):
    system_prompt = """You are a helpful assistant that analysis a corpus of data and helps the user to find the answer to their question.

    You must follow these rules:

    1. provide short and concise answers
    2. do not provide citations from the corpus of data unless asked for

    """
    separator = "\n\n---\n\n"

    with silent():
        with system():
            lm = gpt + system_prompt

        with user():
            lm = lm + separator + "Consider the following data: \n" + corpus  + separator + user_question

        with assistant():
            lm += gen("response")

Is this still possible to do? Thanks.

[slundberg]

Set echo to False:

gpt = models.OpenAI("gpt-3.5-turbo-1106", api_key=os.getenv("OPENAI_KEY"), echo=False)

You can also set it later gpt.echo = False

We are working towards better docs, but sorry a few things like this are still hard to find.

[mlashcorp]

Thanks Scott!