For all examples, please show how to get the answer out

[prescod]

Several of your examples show how to set up the chat or completion.

Then they show you what the chat or completion looks like before its executed.

But not how to retrieve an answer from the chat or completion.

[Harsha-Nori]

Hey @prescod,

LM objects store the outputs of functions like gen and select and can be accessed through traditional [] indexing via __getitem__. Any time you name the output, you can then retrieve the generated portion from the generated portion, eg this example from our README:

# capture our selection under the name 'answer'
lm = llama2 + f"Do you want a joke or a poem? A {select(['joke', 'poem'], name='answer')}.\n"

# make a choice based on the model's previous selection
if lm["answer"] == "joke":
    lm += f"Here is a one-line joke about cats: " + gen('output', stop='\n')
else:
    lm += f"Here is a one-line poem about dogs: " + gen('output', stop='\n')

Note how we assign the output of the select to the variable answer, and then retrieve it later with lm["answer"]. You can also do this with gen by passing an arg to gen (defaulted as the first arg):

lm = llama2 + f'''\
Do you want a joke or a poem? A {select(['joke', 'poem'])}.
Okay, here is a one-liner: "{gen('oneliner', stop='"')}"

lm["oneliner"]

For more complex expressions or grammars, you can assign a name to them using the capture function, which lets you assign a range of values to store as a variable (see how tool_args is used here: https://github.com/guidance-ai/guidance#automatic-interleaving-of-control-and-generation-tool-use)

I should also note that you can always access the current state of the LM -- everything appended, generated, etc. -- by calling str(lm) at any point.

Hope this helps!

[alexmadey-oc]

@Harsha-Nori when I try to use capture I get this error: AttributeError: 'StatefulFunction' object has no attribute 'nullable'. Capture only seems to work when I define a function like @guidance(stateless=True). Without the stateless=True it gives this error. Do you know any way to capture the result of a stateful function? If I try to make my function stateless if has something like {{G|139636300580112|G}} wherever there is a generation supposed to happen.

[prescod]

Hello @Harsha-Nori . thank you. I had figured it out because I was a user of the old guidance.

I was trying to point out that the new guidance documentation usually does not capture the output variables which is confusing because why would you call an LLM and discard the result? The only reason to call it is (usually) to get the result.

Every example should end in:

print(lm["result"])

That would be more idiomatic, clear and helpful.

[jeanibarz]

Just went through the tutorial - super helpful, thanks! But I'm also a bit fuzzy on a couple of things:

• What happens if we don't name the outputs?
• And does naming (or not naming) outputs change how the LLM state gets updated?

Would be awesome to clear this up in the tutorial to help out us newbies. Cheers!

[kaosbeat]

I cannot find a proper way to get an answer out My current workaround, which I use to export a JSON object I generated in a function to the state is:

    ob = {
        "id": 1,
        "text": lm += gen("a joke")
     }

    lm += gen(name="joke"+str(id), max_tokens=0) 
    lm = lm.set("joke"+str(id), json.dumps(ob))

I can then access it from the returned lm object. I could find no other way to return an object generated within an @guidance construct. Of course I then have to do ajson.loads(lm["joke1"])which feels silly cause I just encoded it