programatic query creation

[CircArgs]

Is there a way or intention to create a way to create queries in a programatic pythonic way instead of the dsl like with a builder pattern or something else?

My reason for wanting such a thing is because the only way I can think of within the confines of the DSL to add dynamic control flow is to use python metaprogramming with strings and exec which is ugly, error prone and risky.

consider the example from the homepage:

import re
from lmql.demo import gsm8k_samples

def calc(expr):

      expr = re.sub(r"[^0-9+\-*/().]", "", expr)
      return eval(expr)

argmax(openai_chunksize=64, max_len=2048)
      QUESTION = "Josh decides to try flipping
      a house.  He buys a house for $80,000
      and then puts in $50,000 in repairs.
      This increased the value of the
      house by 150%.  How much profit did
      he make?"
      # few shot samples
      "{gsm8k_samples()}"
      # prompt template
      "Q: {QUESTION}\n"
      "Let's think step by step.\n"
      for i in range(4):
         "[REASON_OR_CALC]"
         if REASON_OR_CALC.endswith("<<"):
            " [EXPR]"
            " {calc(EXPR)
}>>"
         elif REASON_OR_CALC.endswith("So the answer"):
            break
      "is[RESULT]"
from 
      'openai/text-davinci-003'
where
      STOPS_AT(REASON_OR_CALC, "<<") and
      STOPS_AT(EXPR, "=") and
      STOPS_AT(REASON_OR_CALC, "So the answer")

this prompt could be made to use many different versions of calc and each version of calc might require different generation paths/variables and this would require more branches - potentially a branch for each version of calc. How might you dynamically add these branches to the prompt? Also, how would you account for different variables introduced in each branch possibly needing different criteria?

[lbeurerkellner]

Thanks for raising this, I think it is a good suggestion. A builder pattern could be an option, although I suspect it will be a bit awkward if you also want to use control flow. I am open though, to be convinced otherwise, e.g. with imagined example code.

Regarding the example, I agree that different calc variants/tools could lead to a lot of branching. The solution for this, I think however, is not necessarily meta programming. Instead I imagine that type constraints (as currently in the works) could help a lot here. E.g. instead of parsing different tool inputs as multiple variables, you just specify a single [ARGS] variable with a constraints type(ARGS) is ToolParameters. Although, even then, branching based on model-selected tool will be necessary.

On the other side, the meta programming idea is actually kind of interesting if we consider LMQL as a possible intermediate representation for LLM apps.

Very interested to hear your thoughts about a meta-programming interface, but also about how we could extend the core language with new constructs, to implement the example above with multiple tools.

[CircArgs]

A similar problem with control flow is faced by actually a very similar (albeit more "primitive" involving no LLMs) scenario as lmql - parser combinators. I think parsy is fairly elegant in how it is only a few hundred lines but the simplicity and clarity from chaining different parsing rules together is extremely powerful. I could see some similar methodology being used in lmql.

It is possible in parsy to also define arbitrary generators to handle any arbitrary control flow you could hope for and I believe this could be done similarly for lmql if I understand correctly.

As for the metaprogramming, I start with my query function without a decorator like:

async def f(s: str):
    """
    argmax
        "My prompt"
        {some_meta}
        "{{s}}"
        {possible_control_flow}
    from...
    """

and then I do something like:

        source = (
            "async def _f"
            + str(inspect.signature(f))
            + ":\n"
            + ("    '''" + f.__doc__.format(**meta_params) + "\n    '''")
        )
        exec(source)
        SOURCE_PATCH[locals().get("_f")] = source
        f = lmql.query(locals().get("_f"))

where the SOURCE_PATCH partners with a monkey patch for inspect to get the source lines of my _f for lmql since it is not in a file as inspect wants.

Ultimately I get something like:

async def f(s: str):
    """
    argmax
        "My prompt"
        meta stuff
        "{{s}}"
        injected control flow
    from...
    """