Role blocks fail silently inside of stateless grammars

[hudson-ai]

The bug Role blocks fail silently inside of stateless grammars -- I think it's okay if they are only supported in stateful contexts, but the user should get an error.

To Reproduce

import guidance
@guidance(stateless=True)
def foo(lm):
    with guidance.user():
        lm += "hello!"
    return lm
model = guidance.models.Mock()
model += foo()
str(model)
>> `hello!`

System info (please complete the following information):

• OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): Windows 11
• Guidance Version (guidance.__version__): 0.1.15

[hudson-ai]

Same with the block context manager. I know that this may be impossible to implement with the current design for these context managers, but it's worth thinking about.

[Harsha-Nori]

wow great catch, had no idea these were failing!

[hudson-ai]

@nking-1 and I were deep in the trenches yesterday 😂

[nking-1]

@hudson-ai do you remember where the root cause of the bug was? Is it in __add__() in model.py?

[hudson-ai]

It's sort of inherent to the implementation of context blocks. Opening/closing them just adds/removes something from a class-level dict on Model that affects the __add__ method on Model instances.

The problem is that we never hit that __add__ inside of stateless grammars, because no Model instances ever gets passed to the function. They are "compiled" by passing an empty string in the lm slot of the signature.

A workaround may be to send an instance of a "fake" Model subclass through rather than the empty string, but I think it may honestly be worth taking this opportunity to rethink the context managers a bit.

[michael-newsrx]

See also: https://github.com/guidance-ai/guidance/discussions/786#discussion-6587630